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Nil LDR B.D E. KAT2Z2ELUL, called as a witness 
on behalf of the defendants, being first duly sworn, 
testified as follows: 

DIRECT EXAMINATION 

BY MS. OMANSKY: 

Q Dr. Katzell, what is your profession? 

A I am a certified psychologist in New York State, 
and my field of work is Measurement and Evaluation. 

Q What is your educational background? 

A I have bachelor's and master's degrees from 
Syracuse University in psychology and a Ph.D. from Columbia 
University in Measurement and Evaluation. 

Q Do you belong to any professional associations or 
Organizations, and, if sO, what are they? 

A The American Psychological Association; and within 
that the Division of Industrial and Organizational 
Psychology; the Eastern Psychological Association. 

I am a former president of the Personnel Division 
of the New York State Psychological Association; I belong 
to the Association for Measurement and Evaluation and 
Guidance; the National Vocational Guidance Association; 
the National Council on Measurement and Education; the 
Health Professions Interest Group of the American Educational 


Research Association. 
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Those are the major ones. 

Q What has been your professional experience? 
A Right after I gct out of college I worked as 
a children's social worker in Syracuse for six months. 

After that I was a graduate assistant and assistant 
to the director of the Psychological Services Center at 
Syracuse University where I was involved in counselling 
and testing and research. 

I worked with the Cooperative Test SErvice, which 
is one of the predecessors of the Educational Testing 
Service for one summer while I was a graduate student. 

I came to New York in 1952 as supervisor of 
employee testing at Macy's, where I was responsible for the 
employment testing and revising and developing tests there. 

In 1953 I joinec he staff of the National League 
for Nursing, where I remained until 1973, starting out as 
a supervisor of test services to schools of nursing, and 
moving up through the ranks to become director of the 
division of Measurement and Evaluation. 

And then in 1973, January, I became vice president 
of the Professional Examination Service, a private, 
non-profit corporation here in New York City that develops 


examinations. 


I am currently functioning as an independent 
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consultant in measurement and evaluation. 
THE COURT: Are you still the vice president of 
the service? 
THE WITNESS: No, I am not. I left that position. 
THE COURT: Until recently, I take it? 
THE WITNESS: Until September, as a matter of 
fact. 
BY MS. OMANSKY: 

Q IT am trying to recall, it Seems to me that you 
Stated that you had some connection with the State with 
regard to certification. I just have a note of that here. 

A Only that I am certified in New York State and 
I understand I've been nominated to serve but have not yet 
been appointed to serve on the Board of Examiners for 
New York State. 

THE COURT: Board of Examiners of what? 
THE WITNESS: Of psychologists, for certification. 

Q What is the nature of the services performed by 
the Professional Examination Service?? 

A It is called PES; and if I may use the abbreviation 
as the rest of the witnesses have, PES wax established in 
1942 in response to federal legislation, which required the 
development of tests for merit systems; and PES at tha time 


was part of the American Public Health Association, and 
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they were established in order to deveiop tests that would 
be sil by State Merit Systems in the placement, selecti 
of various health workers. 

They have since that time broadened it to health 
related and welfare occupations, but their work is primarily 
involved in developing examinations that have provided for 
State Merit Systems, public health nurses, environmental 
sanitariums and a variety of occupations with which states, 
municipalities and counties are concerned. 


The also developed certification, licensure 


certification examinations for such occupations as social 


work for some states; psychology for most states; 
veterinary medicine; osteopathic surgery, and a wide 
variety, obviously, of health and related occupations. 

While I was there I was directly responsible for 
examinations in the area of clinical chemistry. 

I worked with occupational health nursing; I worked 
with osteopathic surgery and oversaw the work of other staff 
people who were working in the other tests. They had direct | 
responsibility to me. 

Q In part, because I am curious, for nurses, is there 
an entry level examination and then a later examination? 


A The National League for Nursing provides a 


very wide variety of test services in the area of nursing. 


| 
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13 
by all states in the union and were for some years also used 
14 
by several of the Canadian provinces. 
15 
So that all of these were developed within the 


We provide entrance exaw: that were used by both 
registered nurses and pract cal nurse programs throughout the 
cou: try on a voluntary basis by the schools. They were 
competing services. 

We provided achievement, tests in subject matters 
that were used by schools of professional nursing and practica} 
nursing. Some of these were considered to be course 
end tests; other were comprehensive exams designed to be 


given just before graduation. 


Then we prepared and provided the licensing exams 


for both ragistered nurse and practical nurse, which I use 


divisions of which I was directly responsible. 

Q Have you published any professional material? 

A During the t..> I was at the National League for 
Nursing I was responsible for five or six years for a 
monthly article that appeared in one of the nursing journals, 
Nursing Outlook, under the title "Let's Examine," ard each 

month we dealt with a different topic or problem or "esea ~ch 
eport in the area of : measurement and evaluation in nursing. 
Some of them were very general; some of them were 


specific to nursing educution problems. 
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I have also published in the Journal of Applied 

Psycholegy, an article on expectations and withdrawals from 
nocis of nursing. 

In the Canadian Nurse Journal I had articles that 
related to test services in nursing both at the levels 
of schools and at the level of licensure. 

I have had articles in the American Journal of 
Nursing, in Nursing Research. 

I co-authored an article with my husband 
in the Journal of the American Educational Research 
Associa.iion on personality tests, and I was the senior 
editor of a beok that a»peared in 1972 entitled Women in the 
Work Force. 

MS. OMANSKY: I move that Dr. Katzell be admitted 
as an expert in the area of -- a psychometric expert -~ would 


that be correct? 


MR. MINTZ: We have no objection to that. 


I can hardly do that since I at one point considered consulting 


Dr. Katzell myself. 

THE COURT: Some day that happy time is going to 
come where both sides will agree on the expert and get one 
opinion. 


BY MS. OMANSKY: 


Q I might ask, Dr. Katzell, have you ever constructed 
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an examination? 
A Yes. I have constructed an employment examination 
used for aids in hospital settings. 

I have carried primary responsibility for the 
construction of several examinations. 

In these instances it has been primarily my 
role to put together an examination to satisfy a test plan 
or biueprint that has been developed cooperatively or by 
others, and to work with materials which I might develop, 
or which might be submitted to me by experts who are 
oearticularly knowledgeable in a content area to which an 
examination related. 

In those cases my function would be primarily to 
do the psychometric review and revision of the test 
questions if I did not write them myself. 

THE COURT: In how many instances was it that 
you worked in one way or another on the cons*r ction of a 
test, Doctor, approximately? 

THE WITNESS: 20,. ‘ei. 
Q Now, when was your first contact with the City of 
New York with regard to the examinations of Supervising 
Human Resources Specialist? 


A From my calendar I see it was November 26th. 


Q What did you agree to do at that time? 


404 


Slbr 14 Katzell-direct 

A I agreed to loo over the two examinations in 
question and the test analysis on which they had been 
based, and advised your office as to any issues or problems 
that I saw. 

Q Did you agree to testify for us at that time? 

A No, I did not. I think it was not until quite a 
bit later in December that I agreed to testify. 

THE COURT: When you talk about the test analysis, 


is that the job audit that we have been talking ebout in 


this case? 


MS. OMANSKY: Yes. 
Q My next question is, what materials did you have to 
work with? 
A I received a copy of the two exams, 1631 and 2013; 


the announcements that were posted for each of those and 
the task analysis on which the test plans had been based. 

THE COURT: You say test analysis or task? 

THE WITNESS: The task analysis on which the test 
Plan was based. 

THE COURT: You got the test plan? 

THE WITNESS: Yes. It was attached to the examina- 


tion. 


Q When you looked at the materials that you had, 


how did our job analysis compare with your job analysis? 


~] 
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A There was considerably more information there than 
I was accustomed to finding when I had to work on tests 
based on job analyses, and it impressed me with its complete- 
ness. 

Q Is it important to know the competency of the person | 
who originally performed the job analysis? 

A Well, I would like to know that the person was 
competent, and I would want to know something about their 
ability to do it, because a job anaiysis couid look adequate 


even though it omitted points that one would want to have 


covered. 
Q Do you know who prepared the job analysis? 
A Mr. Rosenberg. 
Q Did you have an occasion to determine his competency, 


and, if so, how? | 
A I met Mr. Rosenberg early in December, and I 

raised questions with him that had arisen when I reviewed 

the job analysis, and learned from him of his years of exper- 
ience, I believe it was 15 years of experience, and the 
thousand of job orders and analyses for which he has been 
responsible, and the classification of many jobs in terms 

of these, and, certainly from his experience in doing this 


he would seem to be competent, and he said the right things in 


the right way that one would think he was competent. 
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Q Is it an essntial that a person who does a job 
analysis have formal educaticnal training in test con- 
struction? 

A I would saynot. “oO my knowledge most of the job 
analyses with which I have had to work in test construction 
were prepared by people who had virtually no experience or 
training in test development. 

I think it would be helpful to the person developing 
the test if the person doing the job analysis had some 
awareness of how his output served as input for the test 
construction. 

But in terms of knowing how to construct a test, 
in my experience we are becoming so specialized that the job 
analyst usually knows little about test construction, 
and the test consiiuctor may know very little except text 
book learning about how a job analysis is done. 

I have never done a job analysis. I have seen many 
of them; I have read about them; I have studied about them, 
but I have never done one. 

Q Could you do a job analysis for a test for psycholo- 
gists? 

A I would need to do some homework before I set out to 
do it, but I think I could. 


Q Now, you said that after you had been given the 
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documents and looked them over and you met with Mr. Rosenberg 
you said that you had some questions and that you discussed 
some of those questions, and how they were resolved, if 

they were resolved, and so forth. 

A The first thing that I reacted to was the fact 
that Mr. Rosenberg indicated the names of four individuals 
in four different slots whom he had observed and interviewed 
together for information about the jobs they were doing, 
how they do it; and I mentioned in conversation with Mr. 
Rosenberg that at that time, at the time these orders were 
done, there were only nine individuals holding the title 
of Permanent Supervising HRS; so that the fact that he 
observed or interviewed four of the nine represented almost 
a 50 per cent sample of the population that existed. 

I learned that it is city policy to base these 
analyses on permanent employees and not on provisionals. 

THE COURT: You were aware that there were an awful 
lt of provisionals, so the percentage would be nothing like 
that. 

THE WITNESS: I was not aware of how many pro- 
visionals at that time, but in the course of that conver- 
sation I learned that there were many provisionals. 

I further discussed with Mr. Rosenberg the 


people and the categories in which they dealt with whom 


25 
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he had verified the information that he got from the in- 
cumbents, and they represented the peers and suneriors of 
those ~~~ ~ubordinates -- no, peers and superiors of. those 
whowere in the positions, and in a variety of relationships 
to the incumbents, and from them he got information about 
additional tasks, so that the material he presented listed 
many tasks, I would guess there might be 25 or 30 different 
tasks, and I discussed with him how he moved from the 
itemizing of taks to determining tne categories that he 
specified at the endof the task analysis -- 

Q Are you talking about the knowledge - 

A The knowledge and skills categories which then 
became the basis of the test outline or test plan. 

And he indicated that it had to do with the fre- 
quency of occurrence of these various activities, such 
things as community relations and awareness of the current 
status of activities, department, organization, and so on 
of HRA; things having to do with interpreting written material|, 
preparing written material, budget, management supervision, 
all of these things which one could see from the list of 
tasks were fairly evidence, and they were reflected 
then in the list of the knowledges and skills; and those, 


in turn, became the basis for the test plan. 


And I learned from Mr. Rosenberg that the relative 
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weights that were assigned to these knowledges and skills in 
the test plan were derived from the frequency with which 
the relative aspects occurred in his analysis of the jobs. 
All of this is the way it is described in the 

books 

Q Did you reach any conclusion about the adequacy of 
the job analysis and test plan as laying a proper foundation 
of preparing an examination for Supervising Human Resources 
Specialist? 

A From my experience it was much better done than 
many with which I had to work; it was more complete, the 
end product, the test plan were typical of test plans where 


knowledges and skills were categorized and proportional 


weights are listed to guide the person developing the examinay 


tion. 
Q Based on your knowledge of professional standards, 


how did our job analysis stack up? 


A I thought it was a good one. 

Q Consistent with professiona] standards? 

A Better than consistent I think I would have to say. 
Q Now, did you have an opportunity to examine the 


examinations, numbers 2013 and 1631? 
A Yes, I did. 


Q Strictly from a psychometric viewpoint, what con- 


to 


gi 
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Clusions did you reach about the construction of the 
items? 

THE COURT: Well, we had one of these problems 
yesterday, and one of the reporters was kind enough over- 
night to fish out the definition of "psychc.wzetric" for me. 

I would like to have the doctor tell me what she 
means when she says from a psychometric point of view. 

MS. OMANSKY: I said that. 

THE COURT: Well, what she means by answering that 
question. 

THE WITNESS: Metric meaning measurement and 
psycho referring to psychological measurement, and psychometri 
Standards for a test would refer to those standards that the 
profession of psychology has set down for it to guide itself 
in terms of adequacy of examinations; the wording of 
questions; they might get into statistical aspects; norms 
if a test is normed -- all of these things that have to do 
with characteristics of adequate tests. 

THE COURT: Thank you. 

THE WITNESS: And we use the term "psychological 
tests" very broadly and not in terms of clinical person- 
ality aspects only. A psychological test is much broader 
than that. 


THE COURT: All right. 


22 
23 
24 
25 
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Q Well, looking at the examinations, did you reach 
any conclusions of the items? 
A Well, before answering that directly I would have 
to qualify what I am going to say. 
In developing test questions we are guided by 
literally dozens of rules to which we try to adhere. 


THE COURT: 


In your profession or people in 
general? 
THE WITNESS: In the profession. 

For example, we try to avoid wording questions 
in the negative - "Which of the following is not true?” 

We try to avoid "All of the following except one" 
is something or other. 

In fact, we try to avoid putting into the stem 
of a question "of the following." 

We try to word the question so that when you finish 
reading the stem it is evident to you what answer you are 


looking for. The premise sets forth the information you 


are seeking. 

The horrible example of a violation of that is 
that if the stem of the question said "an egg is:" - you 
don't know whether they mean scrambled or created by a 
hen or what they mean about the egg. 


If you read the stem you should know what you are 


t 


~ 
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looking for in the answer. 

There are books on these things and pamphlets and 
pages of material, and it is virtually impossible to write 
a quesion that does not violate some rule. 

THE COURT: I take it that the end purpose 
of all these rules is to communicate as clearly as possible? 

THE WITNESS: That is right. And sometimes to 
communicate one thing you have to violate a different rule. 

So that one of the things that concern me in looking 
at the test is the fact that many of the quetions say 
"of the following,” and then set forth the premise. 

But in the case of civil service examinations they 
have been challenged in the past because candidates cited 
that something other than one of the following might have 
been a preferable option, and it wasn't offered. 

So now civil servica examinations tend to say | 
“of the following which is the best answer?" 

So in looking over the exam it was quite evidence 
thatsome o. the rules that one sees in the books had been 
violated, and, as I said at the outset, it is very difficult 
to write a question that does not violate one rule or 


another. 


But there were good options. The options 


were all about equal attractiveness as one read them to the 
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person who knew nothing about the subject matter, and this 
is one of your goals in writing questions, that the person 
who knows the subject will be able to go to the right answer} 
the person who doesn't really know will be guessing in the 
dark among things that look pret+ :qually attractive. 
Some of the items were better than others. 
The items, for example, in the area based on the 
reading paragraphs were quite typical of the kinds of 
questions one finds in many national tests that are trying 
to get at verbal skills, reading comprehension; the items 
on graph snd table interpretation are 4lso quite typical. 
This does not mean they are easy items, but for 
an exam of this sort one is not focusing on an easy exam 
or a totally difficult exam. What you are looking for 


is questions with a range of difficulty. 


If the questions were equally difficult for | 
everybody you wouldn't end up with a list, because everybody | 
would get about the same mark, and you would have a very 
difficult time choosing among the people. 

So you include some easy questions and some moderately 
difficult questions and some very difficult questions for the 
whole group, so that you will get a spread of performance 


in the total population that takes the exam, and thus come 


out with a list in rank order. 
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Q Now, are there any means other than subjective 
judgments in checking whether the examination questions 
were properly constructed? 

A Well, one can look at an item analysis and get 
some information about the way the test was constructed, 
its adequacy. 

Q I believe we did provide you with this -- 


THE COURT: What exhibit number? 


MS. OMANSKY: Plaintiffs’ Exhibit 7. 
THE WITNESS: I have 2013 high and low? 
Q 2013, yes. 
Now, what is the item analysis -- perhaps you could 
redefine what an item analysis is, or this one is, anyhow, 
and tell us what you could observe from the item analysis? 

A In t*s procedure that was followed here, as I 
understand it, the city, to do an item analysis, randomly 
selects 300 papers from the total population, in this case 
2013, and from within that 300 selects the 100 with the 
highest score and the 100 with the lowest score. 


They then take those 200 separately, and for the 


highest 100 determine how many of the hundred selected 
each of the four options for each question. 
And similarly for the low group, how many selected 


each of the four options for each question. 


a 


~ 
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By looking at the item analysis youcan see, for 
example, that in virtually every question -- I believe there | 
is only one exception -- the candidates in the high group 
answered with a greater frequency the right answer than those| 


in the low group. 


That one question is question ll, and it has 
many problems. 

The right aaswer is the right answer, and it has 
been published in the popular press, but the question did not 
work in the way it was intended to work. 

The low scoring group answered it correctly more 
often than the high scoring group, but neither of them 
answered it as frequently as one would by chance. 

THE COURT: What was the question? 

THE WITNESS: That was the one that had to do 
with morale and productivity. 

MS. OMANSKY: I will read the question, your 
Honor. 

THE COURT: Thank you. 

MS. OMANSKY: "Question 11: Recent studies 
of morale and productivity tended to show that (a) the 
correlatior between morale and productivity is rather low; 


(b) High morale is associated with high pro- 


ductivity; 


| 
1} 
| 
| 
| 
| 
| 
| 
| 
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(c) Low morale is associatcd with high productivity; 
| 
(d) Low morale is associated with low productivity. 


THE WITNESS: And the right answer was (a). 
Q You said it was published in the popular press. 


What do you mean by the popular press? 


A I happened to have read an article about it in the 
Times. 

Q Of the New York Times? 

A Yes. 

Q Do you think tha* somebody who was perhaps interested |- 
that this was in the area c.f supervision -- would somebody 


| be likely to pick up tnat kind of item? 


A That was the reason I picked it up, because it 
was of interest and relevant to the work I was doing at the 


time I read it. 


THE COURT: I don't think the doctor purports to 
be an expert on this, does she? 

MS. OMANSKY: But the point is, your Honor, just 
as you pick up in the popular press decisions about law ~- 
I mean the point of it is that it wus not such an academic eer 
that it was never mentioned in the popular press. 

THE COURT: I dont remember the particular 
article in the Times, but are you referring to material such 


as that relating to experiments of the American Telephone 
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Company where they were trying to improve interest in the 
job by getting people -- 
THE WITNESS: Those are job enrichment studies, 
and there are many of them. 
THE COURT: ~~ or union efforts to get away from 
boredom on assembly line jobs and things of that kind? 
THE WITNESS: Right. 
BY MS. OMANSKY: 

Q Would you like to continue? 

A Further in the item analysis c e can see that 
virtually all of the options are, shall we say, working. 
That is, within the high group and the low group there are 
hardly any alternatives that had been offered that somebody 
didn't think was a possible right answer. 

There are very few of those. 
Naturally, there are more in the low group who 


picked some of the wrong options. 


In fact, in the high group there are a few questions 


that were answered correctly by a hundred per cent, I think 
three or four questions of that sort. 

This also is some evidence as to the adequacy of 
the content of the questions, and the fact that all of the 


options are working, as we say. And the fact that by and 


large in the high grour, and I think virtually without 
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exception, better than a chance proportion of the group 
selected the right answer. 

THE COURT: You mean on each question? 

THE WITNESS: On each question. In the high 
group it was typically -- 

THE COURT: Except for question 11? 

THE WITNESS: Except for question 11, which was our 
horrible example. 

And, incidentally, until one compiles these statis- 
tics in any test development, one is likely to find one or 
more questions that are not working t!2 way you wanted them 
to, and this is one of the reasons that you analyze the 
results afterwards. 

So that if the question goes back, as it does in 
many testing companies, if the question goes back into a 
reservoir of questions from which future tests will be 
developed, that you won't use the defective question at least | 
without revising it, or seeing whether the times have 
changed. 

Sometimes a question that was good deteriorates because 
of new developments; sometimes a question thatwas very | 
difficult becomes very easy because of publicity,and things 
of that sort, related to its content. 


So from the item analysis it gave a kind of in- 
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formation that I hoped might be evident about the exam. 
Certainly the high scoring people tended consistently to 
select the right answer, meaning that with the exception 
of question ll the tests were a positive indicator. 
Those who did well on items did well on the test as a whole, 
and question ll was what we call a negative indicator, 
those that did less well did better on the item. 

THE COURT: What was the correct answer to 
UB IS 

THE WITNESS: (a) There is very low correlation 
between the two. It is a complex variable, productivity 
and morale, and there are other things apparently intervened 
which are currently being studied. 
BY MS. OMANSKY: 

Q In what type situlaion would you be interested in 
the correlation between productivity and morale? 

A Practically any working situation where supervisors 
would -- 

Q Supervisory people? 

A -~ supervisors would certainly want to know whether 
the moral for which they have some responsibility is in any 
affecting the productivity of their work groups, whether 
that is something they can control, or, at least, intervene 


in smething else that controls. 
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Q I thin’: this item 11 you said was a horrible 
question -- 

THE COURT: She said it was a horrible example. 

THE WITNESS: Right. 

Actually it is a good question. All of the 
options are attractive. Tnere is clearly a right answer 
but a lot of people don't know it, and the information that 
it is asking about is certainly, in my experience as a 


supervisor, is unrelated to supervision. 


Q So you would just say that the question -- 
A The statistics are very bad. 
Q The statistics are very bad? 


That means it was an awfully hard question? 

A That is mrrect. 

Q And your good people, those thatyou want to pull 
out from these exams, if there were lots of questions like 
these presumably you would not be getting the good people? 

A That is correct. 

Q What w. ight should be given to the opinions of 
persons who take an exam as to job relatedness? 

A Well, in the years thai i worked for National 
League for Nursing, I dealt with large numbers, 
maybe hundreds, maybe thousands, of candidates who had taken 


nurse licensing exams, and I got the impression that 


{ 
F 
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those who failed very frequently cited the exam as being 
unrelated to their education, unrelated to what they were 
doing in their job, or just generally irrelevant. 

Whereas those who did fairly well on the exam were 


more likely to indicate that it was related to what they had 


been taught in school; it was related to what they were doing | 


on the job, and so on. 

So I have rather come to the conclusion that I 
would have to discredit the opinion of people who failed 
an exam in terms of its relevance. I would prefer to get 
the judgment of people who are knowledgeable about the 
job in reviewing the content of an examination for its 
relevance to the job. 

THE COURT: Relating that to the situation we 
actually have here, what weight do you attach to the fact 
that in this particular case -- let‘s take the promotional 
exam, a very substantial number of the people who took the 
exam were actually in the job already, and yet we know 
the plaintiff class, at least, in this case, had a large 
number of failures. 

THE WITNESS: The significance I would attach to it 
I think may be related to the wide variety of positions, 
and I have seen some of the variousposition descriptions 


related to this, and the fact that people may be function- 
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ing somewhat narrowly, and yet are eligible for assignment 
to a general list where they must be knowledgeable in a 
broad category -- 

THE COURT: In other words, somebody may be 
doing quite a good job in MDT, and know nothing or very 
little about ACD or the Social Security Act? 

THE WITNESS: Right. 

Q Would you, in a sense, summarize your opinions 
as to what yu are testifying to? 

A The job analysis on which this plan was based im- 
pressed me with the adequacy, thoroughness and completeness 
with which it was done; providing a far more complete in- 
formation than I have usually been fortunate enough to 
work with in developing tests, and the test itself, without 
being expert in the content, the part of the content with 
which I had familiarity, the questions seems to be well 
constructed, seum to be clearly based on the test plan, 
and the statistics that I was provided seem to substantiate 
that the questions were in general doing what they were 
designed to do for the total group, and it seemed to me 
competent, adequate and better than adequate, better than 


many with which I have had to work. 


THE COURT: When you say the statistics given to you, 


you mean --— 
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THE WITNESS: The item analysis information, yes. 
MS. OMANSKY: No further questions. 
MR. MINTZ: Would you wish me to begin cross 


examination or do you want to start after lunch, your 


Honor? 

THE COURT: I guess it is going to be quite 
extensive? 

MR.MINTZ: It is going to be more than a few 
minutes. 

THE COURT: All right, why don't we break at this 
time. 


We will return at 2 o'clock. 


(Luncheon recess.) 
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AFTERNOON SESSION 
(2:00 p.m.) 
tS Bip Re DP Ce Ba ES Bah DB resumed, 


CROSS EXANIWATION 
BY itR. MINTZ: 

Q Dr. Katzell, in the recital of your own experi- 
ence you gave a number of examples of professional-type 
positions in which you had a principal or some type of 
responsibility for constructing an examination. 

'y recollection and comprehension of those were that they 
were all, I believe all, jobs of a fairly technical 
nature, invoiving a definite body of knowledge -- nursing 
requiring certain knowledte in the health sciences, 

you mentioned chemist, osteopath. Did I understand that 
correctly or were there positions that you designed 

tests of which were generalized=-type jobs? 

A Well, for one thing, you are talking of 
nursing as if it were a specialized job, and nursing 
actually comes into several categories. There are a 
great many levels and a great many specialties in nurs- 
ing, although they are tested on a generalized test. 


THE COURT: Do you think you could define 


nursing as the only occupation not included in a supervisory 


HRS? 
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Tih WITNESS: I didn't come across any 
nurses in that, but there may be some. 

The tests there are related to a very 
general test plan where the individual being tested must 
be familiar with the broad ganut of the content, 
although she may upon receipt of her RN be functioning 
in a more narrowly defined area, such as working ex- 
Cclusively in a pediatric or psychiatric or public health 
setting, things of that sort. But they are tested in 
the broad generalities, 

19] I understand that the test may be broader than 
the job. What I am concerned with is as to whether 
there is a difference where you are dealing with a job 
where most of your skills are of a very subjective sort 
of nature, At least I may be wrong but it seems 
to me this is the kind of job we are dealing with here. 
It is not knowing how to administer a hypodermic, it is 
not knowing how to deal with a child having a particular 
health problem, it is something =-- it is principles of 
supervision and how one applies those, What I am ask- 


ing is if you have had experience in designing tests, 


if you agree with my premise, for jobs of a more subjective 


nature, such as this. 


A Well, I still come back to the fact that nurs- 
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ing is very general. Such as this, I don't know that 


anyone has encountered a joh that is quite as general as 


this. 


@) Given that fact, that there may perhaps be no 
job in existence more broad than that of a supervising iin 
resources specialist, what special difficulties does that 
present for the people involved with constructing a 


test? 


A One starts with a test plan where the categories, 


where the tasks have been categorized, and relates questions 


to those categories, draws questions from a reservoir, 
if that’s the way you are doing it, or writes questions 
to satisfy the test plan, as was done, I understand, in 
this case. 

The test plan includes an awareness of current 
developments in the haman Resources Administration, 
supervisory skills, vocabulary, reading skills, arithmetic 
skills, various components that were created by 
clustering the tasks, and that is not different fron 
other tests on which I have worked. 

THE COURT: In other words, you don't do 
automobile mechanics in this job or paint murals, so you 


can at lease define by exclusion. 


THE WITNESS: Right. 
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Q Did you find that the test follows the test 
plan, tive actual examination? 

A As I indicated, I am not an expert on 
content, but there were some areas it was evident 
that the questions related to particular parts of the 
test plan, such things as the reading and the vocabulary. 
Someone who is not a content expert could identify the 


arithmetic and the graph interpretation things. The 


things that relate to supervision are within my own experi- 
ence and I recognized items in that category. Things 
having to do with current developments could be quite easily 
identified in terms of the content of the questions. 

Q Isn't it as a practical matter true that the 
entire examination is a test of reading comprehension? 

A One couldn't answer the questions if one 
couldn't read them, but one would assume that anyone 
trying to take this examination would be able to read the 
questions. The parts that are specifically reading 
comprehension relate to the way one interprets those two e 
major paragraphs and five questions based on each in 
terms of the ~eaning of individual words within a 
certain context and the interpretation of the total 


paragraph by means of identifying a possible title for : 


the paragraph. It's a typical reading-comprehension- 


ou 
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type question. 

0 You also indicated in your direct, as I re- 
call, that they were fairly easy; at least as far as the 
level of reading comprehension they tested, is that 
correct? 

A I don't believe I said that, but they were not 
among the most difficult questions. 

Q Were there, in your opinion, questions on the 
examination outside of those particular five that re- 
lated to reading comprehension which were in fact more 
difficult to read than that passage -- more difficult to 
read and understani? 

ra There I suppose were individual quest. ns that 
were harder to read than others, but the reading level did 
not strike me as beirg an extraordinary aspect of the test. 
The content seemed to he quite evident what the questions 
were asking for. 

"HE COURT? How do you judge difficulty of 
reading? Is that primarily a question of the -~ 
how can I put it -- the rarity of the nature «:* the words 
tiat are used, the complexity of the words? 

THE WITNESS: There are technical ways of doing 
this in which one determines the number of words in a pass- 


age and the number of syllables and the number of syllables 


co 
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between periods so tnat the length of sentences and 
lengths of words within sentences and lengths of 
sentences within passages are all weighted in a mathe- 
matical formula to determine reading level. I did not 
analyze it on that basis. 

Q To your knowledge, based on your examination 
of the job audit, your conversations with ‘ir. Rosen- 
berg, are you able to determine what level of reading 
ability is required to successfully perform the joh of a 
human resources specialist? 

A Not from precise terms but from the job 
analysis and * conversations with Mr. Rosenberg it would 
appear that the people holding these positions would need 
to be able to read and understand the various communi- 
cation distributed within the administration and in 
many of the instances would need to be able to read con- 
tracts or interpret technical legal materials that 
relate to the functions they are performing. Not 
in all, but again, because these are very varied acsti- 
vities, in many. 

Q Was there any effort that you are aware of 
to do the kind of analysis that you have mentioned of 
word and syllable count to determine the level of 


aifficulty of the materials that a supervising HRS has to 
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read? 

A I did not inquire into that. I have no 
knowledge. 

Q As far as you know, it w3n't done one way or 
the other, you just simply don't know? 

A I don't know one way or the other. I have 
not asked that question. 

Q Are you aware of whether or not the test 
writer subjected the test items to an analysis to de- 
termine -~ of that nature to determine the difficulty 
of reading the test items? 

A I am not aware of whether she did or not. 

Q I don't recall if you mentioned whether or not 
you had any direct conversations with Mrs. Willingham? 

A xX did. ¥ met with her on two oczerions. 

THE COURT: What was the substance of your 
discussions with her? 

THE WITNESS: In the first instance, she 
was present in my first meeting with Ms. Omansky where 
I was asked if I would review the tests,and in the second 
instance it was after I had reviewed the tests when 
I talked with her about sources of test questions and 
procedures that she followed and raised e, tions 
with her that had occurred to me as I re «i: d the tests. 


Q Goirg back to the field audit, ,ou indicated 
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that you, to some degree at least, evaluated “ir. Rosen- 
berg's ability to perform that particular task well o» the 
basis of knowing his past experience in doing desk 


audits and otherwise examining the functions in certain 


jobs. Is that a correct characterization? 
A Essentially. 
Q Is there, to your knowledge, a difference 


between the type of an audit that is done for the pur- 
pose of classification, salary structure, and matters of 
that nature, and the type of job a.alysis which is done 
specifically for the purpose of constructing an 
examination? 

A There may be and there may not be. f have 
functions and situations where the audit done for the 
classification served as the outline for the test, and I 
have functions and situations where the specifi -lysis 
of the components of the job was done because a test was 


to be developed for it. 


Q Is there, to your knowledge, any professional 


recognition of a distinction between those two tasks? 
A In this very specialized area there may 
frequently be distinctions between them but they may be 


combined. There are a variety of patterns that would 


be followed. 
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Q But you yourself have never done a job 
analysis for any purpose? 
A That's correct. 
THE COURT: I take it you don't mean you 
flip a coin as to whether you use one or two but there are 
some Situations in which it is appropriate to do both audit 


at once and some that's not appropriate. 


THE WITNESS: Right. 
Q If that is correct,then you are talking about 
two different things, Is there a difference between 


them or is there not? 


A I wouldn't say that it is inappropriate to 
base the test outline on the desk audit, just categorically 
inappropriate, hut in many instances there has been no 
desk audit when one determines that a test is to be con- | 


structed in relation to a job in whichcase you go in and 


compile the answer. 

THE COURT: I suppose you missed a desk audit 
at one time which has become sumewhat out of date be- 
cause tiigcontent of the job has changed, or something 
like that. In any event, you are satisfied that in 
this particular case it was appropriate to make the kind 


of audit which Mr. Rosenberg made, which was not a desk 


audit? 
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THE WITNESS: That's right. 

THE COURT: And that he was competent to 
make it? 

THE WITNESS: Yes. 


BY MRe MINTZ 


Q You have indicated that you became aware through 


the document itself and through conversations with “tr. 
Rosenberg had direct contact with four persons holding 
the position of supervising human resources specialist, 
which you indicated was almost 50 per cent of those 
permanently appointed. You are aware that it is a 
substantially smaller percentage than those actually per- 
forming the task of a supervising HRS? 

A I indicated that, yes. 

Q When you have a job of this nature, which has 
a great many facets to it, where one person with the same 
title may be doing very different work than someone 
else, are there problems when you meet only a very small 
percentage of those people? Might you not be failing 
to observe certain very important tasks in the job? 

A You might be, but % felt that the fact Mr. 
Rosenberg also met with others who supervised the work 
of the permanent sup-HRSs to check out what additional 


tasks they performed, and since the city has a policy 
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that the analyses are based on permanence, this was an 
entirely consistent and not a very surprising practice. 
He gathered the information hy talking with 
these superiors about the variety of tasks that are 
likely to be involved. 

Q If there were no such policy would it have 


been better, in your opinion, for him to have spoken to 


15 or 20 sup-lIIRSs? 


MS. OMANSKY: I ohject to the question. 
THE COURT: No, I-think that is a proper 
question, I recognize that we have to deal with the 


facts that we have to deal with, but the question is 
whether the product that was produced by that situation 
is up to the best professional standards or nat, or at 
least adequate professional standards. 

Do you understand the question? 

THE WITNESS: Yes. 

A It might have been desirable to have had a 
larger sampling of the total gamut of the types of 
positions that are circumscribed hy this titla. I have 
to say, though, there is a rationale for limiting it to 
permanents because those are the people who are not goliig 
to be challenging it in terms of the examination, but 


there is also a rationale for including it. 
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“HE COURTs We all understand that, hut 
supposing, for example, there had only been one 
permanent, would you have felt that that would have 
been satisfactory? 

THE WITNESS: I get your point. You are 
right. 

Q Going further, is there, to vour knowledge, 
an accepted definition in the field of psychmetrics 
of a critical task or a critical aspect of work? 

A A task which, if incorrectly performed or for 
which the person is incompetent, would constitute really 
an inability to perform a an 

Q And, similarly, a critical skill would be one 
which, if the person lacks it, would render him unfit 
to do the job? 

A Yes. 

Q Do you find in Mr. Rosenberg's field audit 
report any identification of any of the knowledges and 
skills that he outlines as being critical, major im- 
portance, minor importance, or any way quantified? 

A Not in cerms of criticality. There are two 
methods by which these are typically classified, and one 
is in terms of criticality ard another is in terms of 


frequency, and I discussed this with Mr. Rosenberg and 


~ 
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the method that was used here was to classify the 
tasks in terms of frequency in which they were identified 
by the incumbents and the supervisory personnel who relate 
to the incumbents, and that is an acceptable approach. 

Q Do you know how this deteimination of fre- 
quency was made? 

Kh I really don't feel that I can speak to it. 
It was based on the -- I have the impression it was based 
on the numbers of tasks that were cited in the one para~ 
graph at the top of the second page that related to 
specific ones of the categories in the knowledges and 
skills specified at the bottom of the page, so that certain 
things were very frequently related to category 1 and cate- 
gory l then received, I believe it is, 20 per cent of the 
weight of the test. It was done in terms of that sort 
of frequency rather than criticality. 

It is difficult in this type of job to assign 

a criticality weight. It might be easier in one of the 
health occupations whe: ‘ailure to perform correctly 
might result in the demise of the patient, but in social 
services they don't usually believe what we are doing is 
that critical to people, I guess. 

Q Nonetheless there are critical skills. If 


a person was illiterate couldn't we all reasonably assume 


a” 
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he doesn't perform this job? 
A I would think. 
Q That is a fairly obvious one. ran’t it 


possible at least that there are others, some less obvious 
but which could be identified? 

A I would think it would he difficult on a 
general examination for a title of this sort to arrive at 
other critical tasks besides illiteracy that would cut 
across the variety of activities under this title. 

THE COURT: I don't know if you would 
call it an activity. You could call it a quality of per- 
sonality, hut what about the ability to deal with people? 
Is that of special importance in this job as compared, 
for example, to being a researcher in a library, where 
whether you can get alongwith people might not be very 
important. 

THE WITNESS: I am glad they didn't try to 
test for that. It is one of our more difficult prob- 
lems which has not cen very effectively solved. 

() Do you agree with his Honor's proposition 
that that ability is critical to performing this job? 


A I world think that ability to deal with 


people would be very important, yes. 


Q And yet it is something which can't practically 
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he tested for, or at least not very easily? 

A One can draw inferences in terms of people's 
knowledge about such things as supervision, things 
of this sort, but if a person knows about those things 
one can't be sure they will always do them, and probably 


vice versa. 


Q That brings up a very important point. If 
a person in fact knows -- take the example of the horrible 
question -- that morale and productivity are essentially 


unrelated, will that knowledge: aake him a better 
supervisor, or have you any way of knowing whether that 
knowledge will make him a better supervisor than someone 
who doesn't have it? 

A No, I would have no way of *nowing whether it 
would make him a better supervisor. 

I think, if I may point out, individual ques- 
tions on a test of this sort may not reveal any particu- 
lar informationabout the examinee as an individual or 
examinees as a group. It is the constellation of all 
of the questions taken together. And this is true of 
tests in general. — The individual's performance on any 
one question really is rather an irrelevancy. It is 
the clustering of responses, the patterns that emerge from 


responses collectively, that are indicative of the in- 
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dividual's skils and knowledges and abilities. 

Q Is it not true that people may have a great 
deal of a particular knowledge but not be able to apply 
it effectively? 

A Some people, ves. 

Q In this example, is it possible that someone 
might know a good deal about the principles of supervi- 
sion as allegedly tested for in the first 25 questions 
but in fact when they are put in the position of super- 
vising people for any number of reasons, other reasons, 
might not be ahle to apply them and do a very effective 
job of supervision? 

A Some people might. 

Q Is it similarly possible that someone may 
lack the knowledge to correctly answer these questions 
but because of innate personal abilities may be excellent 
or very good or at least competent at the task of working 
with subordinates, delegating responsibility and the 
other things a supervisor must effectively du? 

A There may he such individuals, yes. 

Q Again to take question 11 as an example, when 
you find on your item analysis that kind of pattern 
that only 10 per cent of the top third of your test- 


takers got the correct answer, even though you know it is 
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4 correct answer, would it be considered good practice 
to remove that question from your test either by 
crediting everyone with the correct answer or by simply 
removing the question and not counti:..j it one way or the 
other towards the grade? 

j I think it depends on a great many other 
factors and the particular situation. 

I have worked in test situations where an 
iter, for one reason or another, became defective or 
something was identified, something was wrong with it, 
where it was determined that credit would be given to 
everyone on the question. But generally that has heen 
where it turned out there was no right answer or that 
there were multiple right answers, as a consequence of 
which everyone was given credit for it. 


But in a situation like this, where it was 


simply a difficult question, where there is a right answer 


I wouldn't generally think that the situations in which 
I have worked such a question would he removed from 
a test or everyone given credit for it. It happens to 
be a difficult question, and the questions range in 
difficulty from that level up to 100 per cent getting it 


right. 


Q Yes, v,ut this one is so difficult that the cor- 


- 
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rect answers were less than one would expect from 
chance. If people had to guess randomly you would 
expect 25 per cent of them to get it correct. Even 
in that circumstance yor could see no reason for removing 
this question from consideration towards the score? 
7 I think it would have to depend on a very 
convincing rationale for doing it, hut in general I 
would not immediately suggest removing it from the exam. 
Q The item analysis also shows us that in several 
instances on the top group of test-takers, though there 
yere no others where less than 25 got the correct answer, 
there were somewhere a larger number of that top group 


selected one of the other three aiternatives than the 


so-called correct one. Did you observe that in examining 
it? 

A I recall there were a few such items. 

Q What does that tell you about the item, when 


there are -- when even your top group of test-takers, 
most of them get a wrong answer, not just scattered 
among the other three but a particular wrong answer? 
A There is some fairly commonly held misconception 


even among those who are the better candidates on the 


test. 


I might say that the type of document to 
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which you are referring and which we have been calling 
an item analysis,and which is in effect called an item 
analysis in many quarters, is not what is technically 
known as an item analysis. This is more usually 
called an item tally or a frequency count, something of 
that sort, because it indicates only the number of 
people who chose each of the particular responses. 

The usual item analysis gives you several 
more statistics for each item than this does, and fron 
that one might be able to make some of the inferences 
that I think you are pursuing, but the data available 


doesn't provide that information. 


For example, in many of the item analyses 
that the testing concerns now develop you receive in- 
formation that indicates the average score on the total 
test for those people who selected each of the four 
answers, so that you knu. whether in fact this relation- 


ship holds that they were the better people throughout. 


You also in most item analyses receive in- 
formation on either the biserial correlation or the point 
biserial correlation betweer each of the options and the 
total score. This also gives you some more information. 


You can only guess at it from this. 


Q shese studies were not done in this instance, 
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| 3 A Apparently the computer is not programed here 


2 to your knowl 2dge? 
to do that. 
5 Q They are professionally recognized studies 
6 which are done elsewhere? 
7 A In organizations that make test development 
8 their business, yes. 
9 . Would such an analysis enable you to determine 
10 that a particular question was in fact a negative pre- 
il dictor and that you would get better results if you took 
12 another right an#wer or took two right answers as far | 
as the ability to use your test as a predictive device? 


A What would usually tell me is that I need to 
check with either resource materials or resource people to 
be certain that there is consensus in a content validation 
sense that the keyed answer is in fact the correct answer 
and to be able to document that, and it would seldom 
suggest that something other than the keyed answer should 


be keyed. 


Cne of thepurposes of an item analysis is to 
confirm to you that you have keyed the examination correctly, 
but if you find that a large number of people show some- 
thing other than what you had keyed it doesn’t mean you 


should change your key or what they thought was right, 


9 
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you merely have to satisfy yourself what you keyed a 
the correct answer is in fact supported by expert 
opinion, be it written or consultant opinion. 

Q “hat confuses me about what the purpose of the 
test is. As I understand it, the purpose of the test 
is to determine whether people can successfully perform 
the job, not to determine whether they in fact have 
consulted the right textbooks and therefore have come up 
with the technically correct answers. 

poe3an't the procedure you are suggesting put 
something of a reversal on that? 


a I don't think so. It is a matter of having 


obtained these so-called item analysis statistics, verifying 


with whatever resources were used in developing the 
tests that the test was in fact correctly keyed, and 
I said in my original testimony that the examination 
is devised $2 such a way that the items will be of widely 
varying diffic ity so tlat you will come out with 
list, so that not everyone will fall within five 
points of each other, because otherwise there is no 
ranking and the city has no way of making a selection. 


So the questions are deliberately, they must 


be for any examination of this sort, varying in difficulty 


from some that are very easy to more difficult. 
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THE COURT: May I ask you whether in the 
profession there are any recognized critical percentages 
or criteria of one kind or another of the following 
nature: That is, that you have an exam “n which 
out of the 89 questions as here, let's say, if it were the 
case, that 20 per cent were incorrectly answered, or 39 
Jer cent, or any given percentage were incorrectly 
answered by the most successful group, you automatically 
‘conclude that the test as a whole is suspect? Do I make 
my question clear? 

THE WITNESS: I don*t know that I could 
specify a number, but certainly one expects to find a 
minimum of the questions negatively discriminating, such 
that the people who get the better scores tend to get an 
item wrong, and in my experience usually on the first try 
out of the test if you have 5 to 19 per cent of the 
questions showing this type of questionable statistics, 
either negative or very low correlation between perform- 
ance On the item and performance on the test, you would 
review the question to see if there was something wrong 
with the question, but you might still keep it once 
reviewing it because it was judged tobe useful information 


and the answer that was keyed was the right answer. 


THE COURT: My question isn't directed to one's 
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approach to the salvageability,if you want to call it 
that, of a particular question but to the feel that you 
get about the test as a whole. 

In applying the principles that you have just 
discussed, which seem to me to be applicable to the 
judgment of the test as a whole, as well as a particular 
question, what do you come away from your analysis of this 
test be_ieving? 

THE WITNESS: I believe it does a better 
than average job, especially in view of the fact the 
questions can't be tried out before they are administered. 
There are a minimum of questions that show these low 
frequencies. 

THE COURT: That's what I want to know. 

BY MR. MINTZ: 

Q You stated that the questions vary widely in 
difficulty, and as I understood you the purpose of 
that appears to be that you therefore have a widely 
spaced rank list so that the people in the city 
can select people. Is that » Only purpose of 
varying your questions in difficulty? 

A It is a primary purpose. 


Q Well, then, if instead of a rank list your 


goal was to simply produce a pass-fail list and everybody 
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above a certain level would be hired and everybody below 
would not, there would be no reason to have questions 

of widely varying difficulty? 

A You would still have to have questions of 
varying difficulty or you wouldn't be able to separate cai 
people, If they were all of the same difficulty you 
wouldn't end up with a passing point,because what 
difficulty of level tells you is the proportion of 
people that will get it right. If they are all- 
at the same difficulty level, and say you set 70 per 
cent as the difficulty level, you would end up with 
70 per cent as the most frequent score. Everybody 
would be at that score and you wouldn't have any assist- 
ance in making your decision. 

Q I have difficulty in understanding that. r 


don't know if you are familiar with bar examinations -- 


| 20 experience, having taken three, and hopefully the court 

21 will take judicial notice, that the questions are more 
| 22 or less comparable -- 

23 THE COURT: Three in the same state or 

2 different states? 
25 MR. MINTZ: Three in different states, 


A IT am not. I 
Q -- but those are pass-fail. It is my 


to 
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I managed to get admitted to three different bars on 
examination. Nach one only once. 

0 Your candidates are not equal, so you if 
you have 80 questions of as nearly equal difficuity as 
possible, but you have a thousand candidates of widely 
varying ability, you are going to get some level who 
pass and some level who fail, wouldn't you. 

A Yes. I might say I don't know how they 
establish the pass-fail point for the bar exam but I am 
familiar with how they establish the pass-fail point on 
many other occupational licensing exams, and it is 
usually in terms of setting a point that will elimirate 
a certain percentage. 


THE COURT: I don't think that is true of 


bar exams, In taw York, anyway. 

MR. MINTZ: We haven't been able to find that 
out. 

THE COURT: I am not sure whether these 


questions go to the heart of the matter thet we have, 
because the question really isn't, it seems to me, 

ti other there are some difficult questions or some simple 
quessions but whether the test as a whole fairly examines 
the knowledge necessary to the job and whether it is job- 


related. That is an observation only. 
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MR. HINTZ: Ny next question may bring out 
what I am trying to reach here. 
BY “MR. MINTZ: 

Q You are aware, as I understand it, that the 
two tests here in issue were very similar, only ten 
questions differed? 

A Yes, sir. 

Q To my knowledge, I gather it s true that there 
was ro item analysis done on the other test, 16313? 

A T am ndOt aware of any. 

Q It was not supplied tc the p::intiffs either, 
and it appears not to exist. 

Are you also aware that in one nature 
of the circumstances here everyone who passed the pro- 
motional test would in fact, presuming they also passed 
the oral, would in fact be appointed to a permanent 
position as a supervising human resources specialist? 

A I do not know that. 

Q Ar you able to say with any degree of certainty 
that anyone who couldn't correctly answer 56 questions 
on either of these tests, but particular’v on 1631, the 
promotional examination, would be unable to perform the 


job of the supervising human resources specialist? 


A I am not able to say that. Testing is an im- 
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perfect science, and on any test that we can devise there 
will be people who might exceed a passing point and fail 
to do an adequate job, and there might be people who 

are doing an adequate job who would fail to reach 

a passing point. 

Tests have -- we have measures of test 
reliability and validity that get at some of these things 
and by the use of the reliability measures we have 
some estimate of the degree of error in our test scores, 
but for the various purposes for which tests are used 
passing points are established and we know that this 
does disservice in both directions. We know that on 
subsequent occasions the same individuals might have 
higher or lower scores, 

Q Are you aware of the fact that the percentage 
of those passing the promotional exam,for which only people 
who were permanently appointed to the next lower level job 
were eligible, was lower than that for the open competitive 


which was eligible to anyone meeting generalized qualifi- 


cations? 
A No, I am not. 
Q Does that knowledge tell you anything about 


these examinations, when you know the people within the 


agency doing at least a related job at the next lower 


ead 
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level could not do as well on that average as people from 


outside? 

A Could you back up and clarify that for me 
again? I am sorry. 

Q The circumstances here show thatthe promotional 


examination, the percentage of those passing was lower 
than the percentage of those passing the open competitive 
examinations. 

The facts in evidence also show that only 
people who were permanent appointees to the rank of 
senior HRS, which is the next lower rank, could take the 
promotional, Many of those peo, 'e who were such 
permanent appointees were in fact provisional supervising 
HRSS. 

THE COURT: So that as I understand it, in 
Other words, there were more people who were actually 
on the job, a higher percentage of those who were 
actually cn the job, who flunked their test than the 
people who were not on the job flunked theirs, 

MR. MINTZ:  , That is the essence of it. 

™S. OMANSKY: That is not in evidence because 
we don't know, really, who took the open competitive 
test. For example, wa know many of -- I don't know 


how many o. the plaintiffs took the open competitive test, 


to 
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hut you can't make the assumption, I have to object 

to that. | 
THLE COURT: I will sustain the objection 


to the extent that unless we do develop something to make 


that assumption more solidly founded I will have to 


disregard any answer to the question. But if that were 
true would it have any significance in your mind as to 


the validity of the tests? 


THE WITNESS: Not as an isolated datum. T 


would need to know more about, you know, the people and 


something about the jobs, but not just as a datum alone. 

MR. MINTZ: I would like to point out to the 
court that that fact is indeed in evidence through 
Plaintiffc" Uxhibit 1 where all of those who are present 
employees of HRA are identified by race, specifically 
that appears on Exhibit 1C, and a count of those who are 
identified hy race shows it is approximately -- 

THE COURT: You don't nead to go on at 


length about that, Mr. Mintz, because the witness has 


indicated that that fact alone, even if true, would not 


be significant in her view. If it becomes significant 


in another connection I am perfectly open to your 


demonstrating that. 


Q What is a concurrent validity study? 
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A A study which correlates performance on a 
test with some quantitative, currently gathered criterion, 
such as a measure of performance, possilbly a score on a 
rating scale, something of that sort, that was gathered 
at the same time as the test was centered, that would be 
concurrent. 

Q Are you aware that a large number of people 
who were performing this job as provisionals vailed the 
examination? 

A I don't know how many. I actually have 
no data on the » imber tha: “sailed, 

THE COURT: Can you be specific about that? 


Does anybody know offhand? 


"MR. MINTZ: I have some difficulty with 
that. Not at my fingertips. 
THE COURT: I suppose the number would be 


of significance, or the percentage. 
MR. MINTZ: We do know that two-thirds of 


those who took the promotional examination failed. 


TIE COURT: Two thirds of the total number? 
MR. MINTZ: Of the total number, yes. 
Forty-three passed and 96 failed. That is very close 


to a two-thirds figure. 


I think the evidence -- well, it is clear that 


a 


* ad 


mda Katzell-cross 454 
all of them were permanent seniors. The evidence indi- 
cates that a substantial number of them were provisional 
sups. 

THE COURT: I am not challenging you, but 
if the witness needs the figure to answer, siie can't 
answer without it. 

Of the 96 that failed, let's assume 60 were 
provisional. Would that mean anything? 

THE WITNESS: What's the question? 


BY MR. MINTZ: 


Q My question simply was were you awere of 
that? 

A No. 

Q Were you aware that the nature of provisional 


appointments under the civil service laws of the city and 
3tate were such that a provisional serves at the pleasure 
of his employer and could be dismissed at any time, 
there is no tenure in the job for competency or any other 
reason? 

A I have become aware of that during the past 
couple of w2eks, yes. 

Q Dows it then suggest to you that in fact, the 
giving of examination 1631 was a rough concurrent 


validity study? 
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A You skipped a step for me somewhere. I am 
sorry. 

9) Well, the test was given to many people 
presently ho.dinz a job as provisionals? 

A YeS, cut what was collected as the criterion 


against which the test was to be concurrently validated? 
You have to come out with a validity coefficient to have 
concurrent validity. It isn’t just armchair judgment 
that these people,just because they are in the job, ipso 
facto that is a criterion fo. concurrent validation, 

You have to come out with a statistic. 

Q It wouldn't be sufficient for you to know 
that they were all performing satisfactorily, at least 
by virtue of the fact that they had not been dismissed? 

A From my experience in management and supervdion, 
and from working with various rating forms, it is typical 
that people get satisfactory ratings, as I ave seen the 
forms that are used in municipal rating. The choice 
is between satisfactory and unsatisfactory, and the person 
who is unsatisfactory wouldn't be there, so that satisfac- 
tory is a very global criterion, and undoubtedly covers 
a very wide range of abilities and skills and knowledges 
in terms of performance. There are some people who are 


just barely satisfactory and there are some people who 
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are absolutely splendidly superb and you wish you had 
thousands of them, And there are people all the way 
between and most people are in between. 

What you need in order to come out with a 
validity cocfficient is some kind of distribution among 
the ratings as you have some kind of distribution across 
the scores, and if you just have lots of people with 
the same rating you don't get any kind of a correlation 
because you have to have some.variability to get a 
correlation. 

Q Isn't it pessible to construct some sort 
of meaningful correlation simply with knowing you have 
a group of pro; .e most of whom we have good reason to 
believe were performing satisfactwrily but which a test 
Says are incompetent to perform the job? We are not 
talking about the range, we are simply talking about this 
measure says they are competent, this measure says they 
are incompetent. 

A Is that in fact, if I may, what the Civil 
Service claims, that by failing that test it means they 
are incompetent? What it is doing is setting up a 
classification list in terms of ranges of performance, 
and that people who know mere than 70 per cent of this 


will qualify to be on these lists. 
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Now, as I have said, tests are imperfect nd 


some of the people who got 69 might have gotten 71 


another time, We would not expect that people who got 
60 would get 80 another time, Tests are not usually 
that unreliable, But occasionally an individual, for 


various reasons, might change his performance r: mark- 
ably. 

THE COURT: Are you implying also that some- 
one who got 60, let's say, might perform satisfactorily -- 
I don't mean scmebody in the sense there can always be 
an exception, but is it quite possitle somebody who got 
a 60 might perform satisfactorily, but since one of the 
chief functions of civil service examinations is to 
spread out people in accordance with their level of 
success on the exam 30 you can choose from them, that 
merely flunking the exam is not inconsistent with the 
possibility of doing the job satisfactorily? 

THE WITNESS: Exactly. 

THE COURT: It just means somebody clse is 
likely tu do it more satisfactorily? 

THE WITNESS: In most instances, 

THE COUR?': At least it provides a number of 


people that the Civil Service can obviously choose as 


having been found in as honest and arm's length a way as 
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possible? 

THE WITNESS: Right. And one of the 
reasons for using paper-and-pencil tests with groups 
as large as you are dealing with here is that it facili- 
tates screening large numbers of paper. The rationale 
for using this kind of a test in 1972, when the other 
type of test was used in the earlier instance, would seem 
to me to be entireiy logical. 

In the earlier instance there were smaller 
numbers to deal with and you could do that sort of 
thing. But where younmed co screen large numbers 


fairly quickly to eetablish lists a paper-and-pencil test 


facilitates doino that, and some people have difficulty 
with paper-and-pencil tests, some people have difficulty 
with interviews. it is the fairest way for dealing 


with the largest number, but there are problems. 


THE COURT: Putting aside the question of 
some people, by which you suggest individuals -- 

THE WITNESS: That's right. 

THE COURT: ~~ has your experience led you 
to believe that there is any serious question as to 
whether paper-and-pencil tests are not only more diffi- 
cult for some people but for some group” of people, 


culturally deprived groups or -- 
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THE WITNESS: They may be for, again, some 
groups within the culturally deprived, and one needs to 
look at them as individuals as well, because they, too, 
show wide variations and individual differences,. and 


there are large groups of them that don't have diffi- 


culty with paper-and-pcncil tests, and there are 

large numbers that do, just as with the school situation 
and law school and the bar exam and so on. In all 

of these things you can find your parallels. 

BY MR. MINTZ: 

Q Do I correctly understand from the statement 
you have just made that if there were an ability here to 
put more time and perhaps even money into the selection 
procedure some better selection device than a multiple 
choice test might be found? 

MS. OMANSKY: I object to tie question. I 


don't think there is any premise for that at all. 


THE COURT: There may or may not have been. 
Mr. Mintz is asking whether there was, He is asking 
whether that is what Dr. Katzell meant by something 
she said. 

MR, MINTZ: That is correct. 

THE COURT: I think he is entitled to that 


much clarification. 
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A I don*t know that I would say it is better. 
Frequently when you are dealing with small numbers of 
people it is easier, more economical and efficient to 
evaluate training and ex perience, some of these things, 
in an interview process or oral exam or some other 
like device than it is to go through all the work of 
developing an objective examination, which is a lot of 
hard work, especially to come out with a good one. 

‘hen, on the other hand, when you are 
dealing with large numbers of people one of the ways of 
facilitating handling them is to use paper-and-pencil “ 
tests that can he objectively scored. So that in 
the books when they set these up in parallel tables 
these are the things that would be opposite each other 
in the tables, the advantages and disadvantages of oral 
exans, essay exams and objective exams, and which type 
of group it might most cenveniently be used with. 

Q Getting back to the point we were discussing 
before, though, there is another factor you may or may 
not be aware of, We have here a provision of law, I 
believe state law, which requires that persons eligible 
for promotion be given an absolutely preference over 
persons who are frem the outside in this situation. 


That is, everyone who passed all parts of 1631 would 
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get the job be"ore anyone who passed 2013. Do you 
understand that fact? 

XN I am hearing it for the first time. 

Q In other words, what would in fact happen here 
is the person who got a bare 70 on the promotion test 
would be appointed ahead of the person who got conceivably 
100 per cent, if there was such a person, on the open 
competitive, 

You were not aware of this either? 

A Ho. 

Q Does that knowledge cause you to change in any 
way your prevoius answer that when what you are aiming 
at is to rank people and to screen out the less qualified, 
if not perhaps the unqualified -- does this set of cir- 
cumstances change that in any way? 

MS. CMANSKY: I am going to object to the 
question on the following grounds: 
In the first place, there is no challenge, 


I think, to the provision of state law which says that 


people get appointed first from a promotional list. 
That obviously emis a different social policy, and I 
don't think that that goes to the issue of job-related- 
ness, © ‘*o the construction of a test, and Dr. Katzeli 


is not hex 2 to evaluate social policies and it seems to 
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me therefore the question is irrelevant. 
THE COURT: I wili sustain the objection 
to the qvestion much as I would be interested in the 


answer. We are not here to decide how the system 


Now, as cto how the tests are to be created, 
whether there ought to be such policy, is not in 
issue, only whecner the policies that there are 


produce legal rewults or not. 


works but whether it works. ea | 


MR. MINTS» Would the court hear plaintiffs’ 


point of view on chat? 

THE COURTS Yes. 

MP. MINTZ2 Our rea) concern here is we have 
bad an absolute cutofs to two-thirds of the people who 
took the pronotiona. 2xamination, many of which are the 
provistonais and the people we represent. We feel 
that that kind of absolute judgment, even if we assume 
the test is job-related -- 

THE COURT: You mean for this discussiorx 

MR. MINTZ: -- if you have a policy the 
promotion people come first and you are gaying ali the 
people below 70 on the promotion exam are undermualified, 
they will never get a chance to get the job, you are 


just not saying that they are less qualified. 
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THE “OURT: What would your question of this 
witness be? 

MR. MINTZ: My question was whether knowledge 
of the fact -- what in fact is happening is that relatively 
low scores on the promotional exam ous the job before the 
high scores on the open compet: tive, whether that changes her 
view of the value of the cutoff score. She seemed to suggest 
previously that the cutoff score was simply a hel;ful test; 
it was allowing you to cutoff the last qualified. 

THE COURT: I den't th ik such question about 
the value of the cutoff .sore would b« pr per. I think 
we would be extending the law if we went into, and I think 
we would be extending bound:~ies of legal discussior if 
we went into what I am about to talk about, which is whether 
the policies in question have discriminatory results, 
but I don't think that Dr. Katzell pretends to have 
any right to an opinion on that particular subject. 

You may be free tc argue, for exarmvle, that a 
T and E exam would have been better here and would have had 
less discriminatory results, or what a cutoff of 70 in the 
situation thatyou are describing has produced or will sara 
duce, but that is not your field of expertise, or rat »ci, 
not Dr. Katzell's, and it is for that reason that I will fit 


sustain the objection. 
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I have recognized, although you may be free to 
argue that, that although that has not been a subject which 
has been decided by the courts, that does not mean it can't | 
be presented by a lawyer arguing his case. 
MINTZ: With due respect, there are questions 
which deal with the impact of a cutoff score as a discrimina- 
tory device. 
TE COURT: All right. 
BY MR. MINTZ: 
Q Has your experience in this field of psychometrics 
brought to your attention any maerial relating to the dis- 
criminatory impact of written examinations, speaking 


particularly of racial and ethnic discriminations? 


A In some settings, yes. 
Q Pardon me? 
A In some settings, yes, not categorically. 


Q Well, what are those settings, and where do you find 
this information? 
A Well, some of the Supreme Court decisions that have 
dealt with the Griggs Duke Power case dealt with testing 
situations and unfairness. 
There have been studies that were supported by 


the Ford Foundation and conducted at New York University 


that dealt with fairness and validity in testing as it 
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applied to minority groups. 

And there is a fair amount appearing in psychologica 
literature. One can hardly avoid it. But there are 
situations where tests are being demonstrated to be fair 
and being validly usec. 

So this is why I said one cannot categorically 
say that a test, ipso facto, is discriminating against minor- 
ity groups. 

Q Are you aware of the fact that the evidence in this 
case does show that the performance of black candidates, 
at least insofar as the scores achieved, was, on a percentage 
basis, lower than that of white candidates in these examina- 
tions? 

A I don't know what the percentages were, bv% I in- 
ferred, since the case was in existence, that that was the 


situation. 


Q And similarly with regard to Hispanic and white 


candidates? 
A Yes. 
Q Now, are you aware of any means available to the 


test constructor to attempt to minimize any such racial or 
ethnic impact? 
THE COURT: You mean the impact which occrs 


merely because you have a written test? 
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MR. MINTZ: Well, either merely or a particular 
written test. 

THE COURT: Are you talking about how to minimize 
the impact within a written test? 

MR. MINTZ: Yes, 

THE COURT: Once you decide to give one, how do you 
go about it? 

MR. MINTZ: Yes, my question at this point is 
directed to that. 

A Well, one way, of course, is in terms of content 
validation. having the test reviewed by people, or have 
people work on it, contribute the items, and so on, to 
get familiar with ‘he content and judge, in their expertise, 
that this content is relevant to the job. 

In that case it can usually be inferred, a‘though the 
EOC, I understand, leans toward criterion related validity 
rather than content validity, but for many tests i: is 
acceptable, and they accept it, that experts validate that 
the content is relevant to the job, and on that basis iccept 
the content as being suitable for ‘:clusion in the test. 

Now, beyond that one may have special reviewers 
in some situations where you would have people representing 


various ethnic or educational levels to review a test in 


terms of its difficulty for them to read, but this could also 
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be applied to any test. 

One might consult consultants for a variety of 
expertise in this skill, but it is certainly not always 
done. 

Q Well, assuming that you have a good determination 
of the necessary content, are there ways of eliciting 


familiarity with that content wich may be more or less 


discriminatory? 
A I am sure there areways that would be more or less 
discriminatory. 
THE COURT: Than eacn other? 
THE WITNESS: Right. 
THE COURT: Some ways would be more and some 


ways would be less? 
THE WITNESS: Right. 
BY MR. MINTS : 

Q For example, assuming feasibility, if you could 
place someone in a simulated job situation and judge their 
performance in that, would you have any reason to believe 
that that would be more or less discriminatory than having 
them answer written questions relating to the performance in 
that job situaticn? 

A I think I would be inclined to lean toward the writt 


quastions in a situation like that. 
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THE COURT: As being less likely to produce 
discrimination? 

THE WITNESS: That is right, because one of the 


values of objective type tests, of which this is an example, 


is that everyone is exposed to «:e same situation, and 
their nses are judged by the same criteria, namely, 
an ob}. >t ve key, and either they have got it or they 
haven't got it; whereas if you have judges rating people's 
performance in an observed setting, and you have all sorts 
of subjective factors entering into it, it might be that 
the ethnicity of the rater influences the effect of his 
rating on the perso. »eing observed. There are many more 
variables that it is impossible to control, in my judgment, 
in that observed behavior setting than in this. 

Also in the observed behavior setting you are very 
limited in the numbers of kinds of things you can sample. 
Here in a relatively short period of time you can take 80 
samples of a person's knowledge, and this is another one of 
the advantages of objective type questions. 

Q Did I understand you to say that it is impossible 
to objectivize the performance evaluation? 

A Well, you can't be certain that it is totally 
objective. You can train your raters; you can provide them 


with an objective rating form, and you can still have some- 
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thing about the person they are observing create a "halo 
effect" so that they rate in one way or another on the 
basis of some generalized impression, and that is what 
it means. 
THE COURT: I think that Dr. Katzell deserves a 
little rest for the time being, and if she does not, I do. 
We will take a 10-minute recess. 
(Recess.) 
THE COURT: Do you have any other witnesses 
today? 
MS. OMANSKY: Yes. 
MR.MINTZ: I have only two or three more questicns. 
THE COURT: Good. 
BY MR. MINTZ: 
Q Do you have a copy of the job analysis with you, 
Defendants' Exhibit F? 
A This is thememorandum dated February 4th? 
Q That is correct. 
A Yes. 
Q Is there anything in there that shows -- 
THE COURT: This is the job analysis, not the 
item analysis? 
MR. MINTZ: Yes, what is technically called the 


field audit report, Defendants’ Exhibit F. 
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THE COURT: Yes. J remember. 

Q Is there anything in there which shows how 
specialized the position of supervising human resources special- 
ist may be; that is, how unique one person's job might be 
from that of another? 

A In this I would say no 

In other words, it stays only with the commonality 
of che positions and not the specialized functions? 

A Well, in the task at the top of page 2 I inferred, 
perhaps inccxrectly, but I inferred from the variety of 
tasks specified there, that these must be drawn from a 
variety of different positions, because it seemed to me 
unlikely, and also recognizing the parenthetical labels 
that go with the Title, that this would represent different 
tasks performed in different parentheticals. 

Not all Supervising HRS must do all these things, 
but these are tasks that are done by Supervising HRS. 

Q You don't infer from this that all people in this 
title will do each of these tasks? 

A By no means. 

Q Is there anything in this which indicates how 
frequently or commonly it occurs that the people will move 


from one speciality or one agency to another? 


A Not in this. 
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Q Is that, in your opinion, something important 
for the test developer to know? 

A How frequently? No. 

Q Well, you said that you understand from this, and 
I gather, at least, that some Supervising HRSs do only some 
of these tasks, and yet they are tested on what purport to 
be the knowledge and skills required to do all of these 
tasks. 

If, in fact, people are slotted to particular 
jobs where they do two thirds of the test but not the 
other third, but they are tested on their skills in all 
of them, would that affect the validity of the examination? 

A No. What are specified here are the knowledges 
and skills that arelmsic to doing any of these tasks, 
cc that in order t> do virtually all of them one has t 
be able to read and comprehend what one reads. 

Tharefore, one of the categories here is skill in 
language usege, including vocabulary and reading compre- 
hension. 

So that is one of the knowledges and skills that 
is required in order to perform, whether they are going to 
be develop TV materials, or writing a newsletter, or what, 
you hav © have some verbal skills, and these are getting 


at those. 
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In all of them, apparently, one has to have 
familiarity with developments in the programs, so that is 
one of the categories of knowledges and skills. 

Q Developments in the programs, you say? 

A Yes, category 2, familiarity with developments and 
categories in the programs. 

Q You were here on Monday when the plaintiffs testified, 
I believe? 

A Yes, I was. 

Q You heard descriptions of their duties, Mr. Jack3on, 
for instance, who works in Personnel. Would you expect 
that in order to successfully do his job as a personnel 
coordinator, he would have to know, for instance, No. l, 
the knowledge or functions of public and private human 
service agencies beyond his own agency? 


A That is not what I have here at No. l. 


Q Looking at the bottom, the list of knowledge and 
skills requ: ad. 
A Resoonsibilities of HRA Central and constituent 
agencies, 
MS. OMANSKY: It is the different between the | 
promotional and the Sup. 
MR.MINTZ: That is the first I h2ard there was any 


distinction. 
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This is very interesting because this is not the 


fact, I had it yesterday; I believe everything else would 


S exhibit on file, and this is the first time I have seen 
4 this document. 
5 MS. OMANSKY: I t!. ak, Mr. Mintz, you introduced 
. in evidence -- 
q THE WITNESS: Well, apparently the only one that is 
; different is the one that Mr. Mintz cited. 
, THE COURT: You can ask the question in any event, 
10 Mr. Mintz. Just read whatever it is. | 
i You are actually reading from an exhibit, is that 
3 | right? 
3 | MR. MINTZ: Yes, I am readingfrom a photocopy of 

| ” wh : I introduced. I have the actual one here. | 
b THE COURT: I am not challenging it. 
sed Whether it is in evidence or not, you can ask the question. 
M So if you just want to ask Dr. Katzell whether she thinks 
18 X is relevant to Mr. Jackson's job -- 

| 19 MR. MINTZ: I am taken aback by discovering at 

| 20 this point in the trial for the first time that there was 
21 apparently a different field audit for the promotion and 
* open competitive exams. | 
* MS. OMANSKY: No, I think I can explain it. | 
2 There is one category -- there is another document; in | 7 
. | 


xX 
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be the same except for Category 1, and I would be perfectly 
happy to irt.roduce the other one into evidence. 

MR. MINTZ: I think appropriately it should be 
introduced as F-1l. 

THE COURT: Let's call it F-1l. I don't pay any 
attention to who introduces the exhibits, anyway, but it 
would make it more logical if it is a similar number. 

So we will mark it Defendants' Exhibit F-1l in 
evidence. 

(Defendants' Exhibit F-1l was received in 

evidence.) 

THE COURT: How are you going to describe it? 

MS. OMANSKY: There ic a slight difference in the 
description cf the subject at the top. 

THE COURT: Let me see it, please. 

(Exhibit handed to Court.) 

THE COURT: For the record, Exhibit F says "Field 
Audit Report, Promotion to Supervising Human Resources 
Specialist, Examination No. 1631 and Open Competitive 
Examination," whereas F-1 does not include the words 
"and open examination." 

All right, we have identified it so that we know 
what we are talking about. 


THE WITNESS: To facilitate things, Mr. Mintz, 


& 
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I do have the wording of the 2013, Section 1, because 
it is in the test plan in the way that you read it, 
functions of relevant public and private agencies. 
I think that a person working in personnel functions 
would certainly need to be aware of the varisus agencies 
he was serving. 

Q Well, Mr. Jackson is assigned to the Agency for 
Child Development, not to Central Management, and performs 
personal work for that agency, not for the entire realm 
of HRA. 

Would your answer still be the same, that he needs 
to know about all the agencies of HRA? 

A I would think he would have to have general 
knowledge of other agencies of HRA. He would need to perhaps 
know, in doing his job, about the agency in which he is 
working, but I don't think it would be inappropriate to 
expect him to know about the agencies in HRA. 

THE COURT: That is primarily because some day he 
might be, for example, in Manpower Development. 
If he never were to leave Child Development, 
would you still feel he would need to know anything about 
Manpower Development? 
A I don't know exactly what all of his duties are in 


Personnel, but simply relating to the people in his agency, 


a 
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I would think it would be helpful. 

THE COURT: When you say "his agency," are you 
referring to the Child Development Agency? 

A Yes, Child Development Agency, to know what goes 
on in the other agencies. 

THE COURT: I take it Mr. Mintz is asking not 
whether he has to know what goes on in Child Development 
Agency but whether he has tolnow what goes on outside of 
the Child Development Agency in the Human Resources 
Administration; is that correct? 

MR. MINTZ: That is correct. 

THE WITNESS: Oh, I would think if he is dealing 
with personnel matters within the Child Development Agency, 
he would reed to know what is going on in other agencies 
in HRA. 

THE COURT: Why? 

THE WITNESS: Rogauce he is dealing with people 
who are dealing with those agencies. 


THE COURT: All right. 


THE WITNESS: Well, apparently many of these people, 
from some of the testimony I heard on Monday, they have to 
be aware of the other resources, and if he is relating to 


them, I think he would need some familiarity. 


THE COURT: I suppose it is a fact that 
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everybody seems to agree that there is -- I won't say 
interchangeability, but a lot of movement from one agency 
to another, would mean that even if you are only a personnel 
man in one of them, it would be desirable for you to 
know about what happens in other agencies, so that if some- 
body comes from their to your agency you will be able to 
cope with it, so to speak. 

THE WITNESS: It is also my understanding that 
the task analysis, part of 1. was approved by the Admin- 


istration in terms of the categories as being appropriate. 


t3 
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Q Simllarly, did ymhear Mr. Yourman testify this 
morning that -- I believe his figure was something in the 
nature of 35 per cent of Supervising HRS's in fact did not 
supervise? 

Do you recall that? 

A Yes. 

THE COURT: He said 60 to65 did, yes. 

I am just explaining to the witness how he said it. 

Q Would that in any way cause you to question an 
-xamination where such heavy weight is given to super- 
vision, 25 questions out of 80, better than 30 per cent? 

A I didn’t understand it was that many, but in any 
case I again beeause of the mobility and the opportunity for 
transfer within the administration, since so many of the 
jobs do: involve supervision, I think it is appropriate 
that a substantial part of the examination involve 
supervision. 

There is none of the categories that is as high 
as 30 per cent. Cagegory 5 ig 20 per cent, and that 
includes staff development, supervision and guidance. 

Q That is true, but I believe that Mra. Willingham 
testified that an examination of the tests would indicate 


those 25 questions are in fact very much related to super- 


vision and staff development, although she suggests they 


_ 
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also relate to other areas. 
A There are many of the questions in here that could 


be and probably are coded in more than one of the cate- 
gories, and with 20 per cent -- if all 20 per cent of 
Category 5 was supervir on that would be 16 questions out 
of 80. Is that right? 

A Yes. 

Q You d= understand, I take it, from the testimony, 
since you don't suggest that you yourself have studied 


the job, that there is some movement of supervising 


HRS from one job to another. Do you know whether that type | 


of movement in fact involves a considerable change in 
function or merely a change of location? For example, 

a person doing personnel with one agency might do personnel 
with Central office or personnel with another agency but 


wouldn't be put out as a coordinator of parent involvement, 


for example. 
A I have no information on that. 
Q You don't know one way or another? 
A No. 


Q Finally, do yo. have any judgment or opinion as to 


whether results on this test are in fact predictive of 


anything? Is a person's score on the test in fact predictive 


of anything? 


21 
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A I couldn't express an opinion on that score. There 
are no data allowing me to express whether it is pre- 
dictive or not. You don't look at a test and guess its 
predictability. I would say it appears to have content 
validity, and that the method by which it w:.s developed woul 
support that ention, but predictability has to be es- 
tablished by statistics. 

Q So you don't know then whether someone who passes th 
test could do a better job than someone who fails or someone 
who gest a 90 could do a better job than someone who gets a 
70? 

A I don't. 

MR. MINTZ: I have no further questions. 

MS. OMANSKY: I would like to first address myself 
to Mr. Mintz’ statement that other courts have given certain 
weight to the automatic cutoff scores. 

I might add that other courts have also thrown 
out tests merely because they have not been predictively 
validated or concurrently validated and of course that is 
not the law in this circuit any more, and most of the 
testimony of course in waded cases attacked exams on the 
grouns that because they had been predictively validated 
therefore they were not proper tests. 


However, that is just getting something off my 
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chest. 

THE COURT: Were you getting off to me or to the 
witness? 


MS. OMANSKY: To you. 
THE COURT: Consider it gotten off. 
REDIRECT EXAMINATION 
BY MS. OMANSKY: 
Q Dr. Katzell, I suppose any test that involves 
reading words is a reading comprehension test, I would 


imagine in a sense? 


A In a sense. 
Q You have to be able to read? 
A If one cannot understand what one reads then one 


cannot answer the questions. 
Q You mentioned a word and syllable count. Is that 

a customary procedure for examinations or is that a rela- 
tively rare procedure? 

A To my knowledge, it is a relatively rare procedure. 
I was involved in only one instance where it was done and that 
had to be done with a test that was being deveiuped at a 
very elementary level for nurse's aides, and we attempted 
to determine the reading level of that test. 


In standardized testing practice I think it would 


pe most typically done when one is developing reading tests 
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for use in public school settings where it is important to 
establish the reading level of the test. 
Q You lost me. 
You talked about some biserial something or other, 
and I wondered how fatal is ic to a test constructor 
like the Department of Civil Service if they did not do a 
biserial something or other? 


A 


Dr. Katzell, you are talking about the item 


analysis, and the item analysis cannot possibly be done un- 
til after the test is constructed, so that the lack of that 
does not affect the way in which the test was constructed. 
The statistics that are sometimes compiled in an item 
analysis are of use after the fact in looking at tie 
results, and certainly one can get considerablé information 
from the item tallies that we have here. The other 
information would add an ingredient. 

Q I suppose that the item analysis might be 
particularly relevant if you were going to use the items 
again? 


A That is right. That is when you would particularly 


Q Do you know, or have you heard, whether it is the 
practice in New York City to use tests more than once? 


A For the reason that the questions are published 
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after they are used they can't he used gain in the same 
form so that there would be no reason to go through the 
@tailed analysis for item revision for a subsequent use. 
Q There was a development of a line of questioning 
that led you to say that it may be that people who answer 
the questions correctly on the test may not do the job, and 
that would be true of this test perhaps. Is it true of 
any other test? 
A I am afraid it is true of all tests, and as Ihave 
said on several occasions it is an imperfect science and 
even a test that comes up with a validity coefficient 


for predictive purposes that is very respectable in psycno- 


mtric terms, that such coefficient at probably around point 
5 would be regarded very well, because it improves your 
prediction, but in a correlation point 5 between a test 
score and some performance criterion you would have very 


many people doing well on a test and purely on a job. It 


is not a perfect predicter such as LSAT does not perfectly 


predict performance in law school. 


THE COURT: And a law school doesn't perfectly 


predict what happens afterward. 
THE WITNESS: That is true. 
THE COURT: They say the A students become -- 


THE WITNESS: CPAs. 
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THE COURT: Something like that. And B students 


become judges and the C students become the wealthy lawyers. 


Q You were present when Mr. Jackson was testifying? 
A Yes. 
Q Do you recall whether or not Mr. Jackson started 


out in the Agency for Child Development or did he start 


out in HRA Central? 


A tT don't remember. 
Q He did start out, I believe -- isn't that correct, 
Mr. Jackson? Didn't you start out in HRA Cenxral? 


MR. JACKSON: Yes. 

Q He started out in HRA Central and then was trans- 
ferred to the Agency For Child Development, and he dealt 
with personnel in both ratters. Do you still say it would 
be important for Mr. Jackson to know the functions of 


HRA Central ~s cc. pared to the functions of ACD? 


A The impression I get from the variety of the 
duties and from having read some of the forms that relate | 
to the jobs, the specific jobs, it would seer. to me to be | 
hard to find one of the jobs in which people didn't need 
to have an awareness of the agencies within HRA. 

Q Do you recall whetner Mr. Jackson mentioned 
whether or not he supervised? He did mention he super- 


vised. I don't know where I am getting. 
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Let.'s see. Just *“o sum up, if you read the cases 
in this area you get the idea that there is a test out 
there waiting .o be discovered which is perfect. Do you 
have any commencs on that, any sum up, final comments on the 
existence of the perfect test? 
A I don't think I will live to see it. 
MS. OMANSKY: Thank you. 


THE COURT: That is a long time to come. Thank 


you very much, Dr. Katzeli. Your testimony was very 


interesting. 
MS. LOREN: Dr. Williams. 
EVERETT BELVIN WILLIAMS, 


called by the defendants, having been first duly sworn, 

testified as follows: 
DIRECT EXAMINATION 
BY MS. LOREN: 

Q What is your profession, Dr. Williams? 

A Psychologist by training. 

Q And what degrees do you hold? 

4 I have a Ph.D. in Psychology from Columbia, two 
masters degrees from Columbia, one in psychology and the other 
in business administration, and a B.A. in psychology and 
philosophy. 


Q To what professional organizations do you belong? 


“ 


~ 
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A I am a member of the American Psychological 
Association, the American Educational Research Association, 
the New York State Psychological) Association, the Association 
of Symbolic Logic -- 
THE COURT: That is good enough for me. 
I don't know how many Mr. Mintz wants. 
What I would like to ask is what portion of 
a psychologist's budget goes to dues in associations. 
THE WITNESS: Considerable, unfortunately. 
A Dr. Williams, san I ask you at this time to 


state your ethnicity for the record? 


A Black. 

Q Do you have any publications? 

A Yes, I do. By name there is an article on deductive 
reasoning and schizophrenia. There is a chapter in a book 


which is an introductory text to psychology published by 


Floyd Drew. 


THE COURT: Is that by yourself or in collabor- 
ation? 
T:.£ WITNESS: The chapter is by myself. 
THE COURT: I see. 
A There are some other articles connotative of 


meaning in driving, which is a description of a testing 


procedure for identifying potentially good and potentially 
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bad drivers. 

THE COURT: Of cars? 

THE WITNESS: Of vehicles, cars, trucks. 

THE COURT: Is it true women are worse drivers 
than men? 

THE WITNESS: No. 

A A more recent paper is one given at the 
American Psychological Association in 1971 on the testing 
of minorities in new opportunities and more recently in 
1972 on the testing of minorities and the problems appli- 
cable thereto. 

Q Where are you presently employed? 

A At the Educational Testing Service, which has a home 
office in Princeton, New Jersey. 

Q What is the Educational Testing Service? 

A It is a non-profit organization that is chartered 
in the State of New York under 501 C.3, in which its primary 
business is one cf test development, test administration, 
test construction, and it offers otehr services to edu- 
cational institutions and to professional organizations ranging 
from dentists to lawyers to mechanics, a local array of 
professionals. 

THE COURT: Those are the SATs and LSATs, 


et cetera? 
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THE WITNESS: Yes, and for business school and 
occupational schools. 

Q What are your responsibilities? 

A I am vice president in charge of operations, which 
includes 7 divisions primarily relating to the task of 
assemblage, distributing, scoring, managing, testing test 
centers, people who are involved in the administering of 
tests, et cetera. 

In addition to that, I am in charge of an edu- 
cational « relopment unit which is concerned with advance 
technology in education, and two corporate responsibilites, 
one in assurring that the testing procedures inETS 
internally are in accord with acceptable professional 
standards as far as ourselves, and accordingly we have to 
monitor the test construction, the test administration, 
and the test interpretation -. policies of the various 
programs. 

Next, I am in charge of a special committee 
related to the assuring that test sites where studies are 
sent to take the various tests mentioned before are free 
of what is called a hostile element that might be hostile 
to them because they are minorities, women, or what have 
you. 


Q What positions did you hold prior to your affili- 
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ation with ETS? 

A I was an associate dean at Teachers College at 
Columbia University, working with the dean in the area of 
administration, in general administration in terms of 
action plans for the college, and so on. 

Prior to that I was a director of the computer 
section at Teachers College in which I served administrative 
functions, the research functions and the student education 
functions. Prior to that I functioned as counselor in two 
deans' offices, one at Hunter's College and the second at 
Columbia College of Columbia University. 

I was also a professor and assistant professor 
and I taught both at Barnard College while being at 
Teachers College. 

Prior to that I was an interne in the Veternas 
Administration programs in the General Medical and Surgical 
Hospital in East Orange, New Jersey, Montrose Psychiatric 
Hospital at Montrose, New York, the Brooklyn Regional 
out-patient clinic in Brooklyn, New York, as a clinical 
psychologist doing testing, psychotherapy, and the rest. 

Q Have you ever been involved in the preparation 
or formulation of guidelines in the area of employment 
testing? 


A Yes, I have 


~ 
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Q What guidelines? 
A For the otfice of Federal Contract Compliance. 


I have been a member of the Ad Hcc Advisory Committee 
whose charge was to review the earlier specifications 
that had been brought into being, to update them, to 
revise them, and finally from them assist in designing two 
books, one for field audit, and the second for personnel 
officers who would be charged with the use of these docu- 
ments. 

THE COURT: For what office? 

THE WITNESS: The office of Federal Contract 
Compliance, which was in the Departmentof Labor. 

A The second was the testing committee for the 
American Psychological Association ofwhich I am a member. 
The charge ther was and is to update the Ame: ican 
Psychological Association standards of test construction and 
administration which happen to be referenced by OFCC 
guidelines and EEDC guidelines. Accordingly, it was not 
necessary to updace them. 

Q Are you familiar with the currently existing 
guidelines of the EEOC on employment testing? 

A Yes. 

Q Are you familiar with the proposed federal uniform 


guidelines on employee selection procedures? 


491 


mdbr 14 Williams-direct 

A Yes, I am. 

Q Have you been asked to comment on these proposed 
guidelines? 

A Yes, I have. 

Q By whom? 

A By representatives from the Civil Service 


Commission and also from EEOC, the legal staff. 


Q On what subjects were you aske Oo comment? 


A We had to review the entire set of proposed guide- 


lines to see whether they faithfully represented the content 


of the former set of guidelines and whether some of the 
proposed revisions to the public sector would do any 
violence to the generality of the guidelines. 
Q Was this the U.S. Civil Service Commission? 
A That ig correct. 
MS. LOREN: Your Honor, I move the admission of 
Dr. Williams as a qualified psychometrician. 
THE COURT: Permission granted. 
Q Would you define content validity as used in 
connection with employment testing? 
A It is a validation strategy in which one is 
concerned in drawing an inference from the results on a 


test and generalizing with skills or personal attributes 


of people. The strategy entails typica)ly a task analysis 
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of some sort where one goes in, and one might do that 
in a variety of ways, but a task analysis. 

Secondly, the establishment of categories or 
in this context it is identified as areas which are relevant 
Or presumably which would contain all the elements of a task 
analysis. That might be done by rational examination or 
by statistical procedure. 

Thirdly, it entails identifying the format that the 
tests should take, which would mean paper and pencil, 
simulation procedure, or some performance activity that could 
be observed. 

Fourthly, it entails the construction of items once 
the format has been decided, the construction of items 
which would go into a sample each of the categories. 

Each of the categories in this context migWf€ require several 
items before you would have some confidence that you were 
getting a reliable measure of the dimention. 


Fifthly, there has to be some confirmation that 


the test is being administered in a way extraneous influericesg 
are not adding error to the results. 

Sixthly, it entails some estimate of the reliability 
of the areas of the clusters that have been included in 
the test. And hopefully you can assume that to get a 


general estimate of the reliability of thetest. 
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Q Is it possible to assess the tent validity of 
an examination after the fact? 


A Yes, it is. 


Q What would a testing expert be evaluating when he dogs 
that? 
A You essentially would have to go back and see 


if the steps that are verbally identified in writing or other 


wise are a relevant step, and, secondly, hopefully go back 
and gt some sub-sample of the data or observations that had 
been called upon or gained by the first set of task 
analyzers and/or test constructors. 
In effect you would retrace those steps and see if 
they made sense yet from another independent vantage point. 
Q Is this the same process that would be involved 
if you were asked to construct a valid test from the 
beginning? 

A Yes, essentially, except it might not be as exten- 
sive as the first. 

Q Could you please describe what you were asked to 
do by defendants for the purpose of this litigation. 

A I was first called by Mrs. Willingham and asked 
if I would consider looking at the procedure that had been 
followed in the valida-tion effort and then comment -- 


THE COURT: When you say the validation effort 
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you mean the validation effort made after the exam was 
administered? 

THE WITNESS: No, I do mean that in content validity 
all the steps taken are central part of the validation 
effort. 

THE COURT: Including what was taken before; in other 
words, the job audit, et cetera? 

THE WITNESS: Yes. Those are essential character- 
istics of a content validation effort. 

A If I would take a look and then give my profession- 
al judywent or share my professional judgment with her 
and other of her colleagues as to whether it had been in 
accord with professional standards and if I thought it had 
been would I be willing to say as much at atrial. She 
didn't ask me about the alternative. 

Q Do you recall when she first contacted you? 

A It was in October when she first talked with me 
by phone. It was at that time that I suggested that she 
might want to contact other people. She called again and I 
consented that I would at least take a look, so she agreed 
to send to me some materials that I could examine, and 
I think they arrived on October 29th, dated October 26th. 

Q What materials did she snd you? 


A I havethem here. It is a copy of the test 
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plan. It is an announcement. It is a copy of the test, with 


a key. And there is another document, I guess it is called 


Experience Form A, It is a notice of examinatica. 
Q Did you agree to testify for the defendants at 
that time? 
A No. I agreed that I would first look and we could 


take the next step to the next question, 


Q ' assume that since you are here after lookina -- 

rN agreed later to testify on December 26th, as I 
recall. 

Q How did you go about determining the content 


validity of this examination? 

A I was really examining the content validity 
procedure. It was arranged through Mrs. Willingham for me 
to interview various people who are employed in HRA. I did 
not know anything about them, their titles, or what have 
you at the time. I talked with each of the people and in 
sme instances were able to observe partially what they were 
doing, and then I talked with the persons who had been re- 
sponsible for the construction of the test, such as Mr, 
Rosenberg, or for at least part of it, and also Mrs. 
Willingham, and I talked to others in the agency. 


Q Was it Mrs. Willingham to choose the people in the 


agency that you were speaking to? 
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A I don't know she chose them. I depended upon her 


for my contact to set up appointment hours and people. 


Q How many people did you interview? 
A No, according to my count. 
Q Could you give us their names and po. tions and 


tell us about what they did? 
A Yes, I can. In fact, I guess the number 9 is not 
entirely correct; it is really 12. 
I interviewed the following persons: Miss or 
Mrs. Iona Pressley, who is in MCDA, a director of personnel; 


Mr. Javier Antompetri, who is in CDA as an acting budget 


director; Mr. Roman Mogilinski -- 


THE COURT: I am not so interested in their names 


as in their positions. 


Q Were any of these people supucvising hjman resources 
specialists? 
A I am not really sure that they were. Some had 


taxen thetests, and some do have that label, yes. Some 
had taken the test and indicated that they had passed it 
for the Sup. level. Some had taken the test and not passed 
it. Some were functioning as supervisors, in fact. Some 
were not who had the label. 


Q How much time did you spend interviewing the people 


in HRA? 


to 
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A Per person it varied, but approximately an hour, 
sometimes longer. I spent four days. There are half days 
in some instances. 

There are five days, but one of those is really a 
half day. 

THE COURT: Am I to understand the others were 
roughly full days? 

THE WITNESS: Yes. 

Q In the course of visiting these different agents 
what did you observe as to the nature of the job of 
supervising human resources specialists? 

A I guess some of the other witnesses have testified 
they are highly variable in terms of what they are doing, 
and ranging from people who had the responsibility of 
policy design and review from persons such as Mrs. Kazajian of 
ACD, who is assistant to the Commissioner, topsople who 
did a lot of report writing, such as Mrs. Cashman, ACD 
supervisor senior planner, to people who were involved with 
personnel and personnel review procedures. There was a 
person in -- I suppose I refer to it as the transportation 
distribution center where a lot of that kind of work 
was done -- to people who were handling the low level budget. 
I call it low level because I am just tallying up figures 


in preparing the report -- budget things, budget reports, 
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to persons such as Mr. Mogilanski, who had sophisticated 
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kinds of tasks such as policy analysis, it is a policy and 
budget sys.em design, which involved looking at a lot of the 
financial aspects of the city and how it related to the 
various agents, and so on. So it was very varied. 

Mr. Stephen Krause was performing more traditional 
fiscal policies of field audits and looking at the correct- 
ness of collection data, and so on. The variability was 
quite remarked. 

Q Did you reach any conclusions s-eyarding the 
adequacy of the job analysis performed by the department of 
Personnel? 

A Yes, I did. It seemed to me, given the kinds of pollicy 
constraints that exist in the agency that it was a very 
well done job of doing both the job analysis and the test 
construction. Under the circumstances I would say it 
was very good. 

THE COURT: When you say given the policy con- 


straints, are you thinking in particular of not interviewing 


professionals? 
THE WITNESS: That is correct. 
Q We have heard testimony that only four out of 


approximately 8 permanent supervising human resources 


specialists were interviewed. Do you have any opinion 


499 
mdbr 22 Williams-direct 
as to the ac yuacy of the sample here? 
A For a total of nine that was very adequate in terms 


of a sample percentage. Under that policy constraint ‘i+ is 
adequate. Under a larger consideration, if one had a 
full domain of people who were incumbents then you 
typically would want to have more observation points if 
there are these wide differences. 

There are rules of thu pout that ir terms of 
how many people you would, or what percentage of the total 
you would interview. ometimes that varies from 10 per cent 
to 25 per cent of the total class. 

In some instances, in some undertakings, the per- 


centage can be much smaller. 


But I would say under these conditions that was 


adequate. 

THE COURT: In any event, whether it would have 
been nicer to interview more people -- not nicer, but more 
effective -- if the policy weren't against it, I understood 


your opinion also to be that you believed the test to be 
adequate, is that right? 

THE WITNESS: That is correct. 

THE COURT: Would your opinion as to the adequacy 
of the test be altered if the policy were changed? 


THE WITNESS: It potentially could. For example, 
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the ceason I say that, in the validation effort 
Mr. Rosenberc and/or others who helped him identified 
eight areas that they felt to be important. It is conceiv- 
able that there would exist a class of employees that were 
doing something quite different, and accordingly one of 
those areas, any number of those areas, is not sufficient 
to cover what he or she is doing. 

THE COURT: You are talking about the possibility 
you might have to add something to the examination? 

THE WITNESS: One might’add and one might con- 
ceivably minimize the importance of. one or enhance it. 
But since you don't know what is out there you can't really 
say with firmness. 

Q Dr. Williams, the position of supervising human 
resources specialist is what is known as a generic title. 
We have had some testimony that perhaps this should be 
broken down into a series of different titles. Based upon 
the interviews you conducted what is your opinion as to the 
appropriateness of the generic title? 

A It seems appropriate just because people are doing 
so many different things, and although I don't know what 
ail the policies of the agency would be, from the information 
that has been given to me it would seem that there are not 


enough people in some of the categories or some of the function 
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areas to be able to justify separate titles, but that 
might be a matter of circumstance. 

At the moment there are insufficient number of 
occupants possibly and the administrative practice of 
having people shift from one place to another probably 
wouldn't add anything to break it down into smaller specialty 
areas. That is if the practice continues where people 
move. In some instances I observed one person who had 
responsibility for some 50 odd people for a short period 
of time and then the next time she was involved in report 
writing with responsibility for no people. With those kinds 
of fluctuations and that kind of fluidity in the system 
it seems to me you are stuck with generic classification. 

Q Do you have any opinion as to whether the three 
specialties that we have had testimony about here in the 
MDT and the management and the community programs would ' 
any more appropriate than the generic title? 

A Any more appropriate? 

Q Yes. 

A My response to whether it is appropriate or not is 
that in some instances in some ways I could argue that 
from what I saw one might want another category that dealt 
only with financial auditors or you mightwant another 


category that dealt with people who dealt with nothing more 
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than just policy planning. They are all different in kind. 

It would be hard for me to say that that would 
be more appropriate. 

As an administrator I would probably have a bias of 
saying I would stick in the generic, but an administrator 
in the city system might have different kinds of problems. 

Q Did you review the open competitive and promotional 
examination in question here? 

A I looked at one copy of it. I am not sure I have 
both copies of the exam. I have one copy of the exam. 

THE COURT: The question is not what you have but 
what you have looked at. 

THE WITNESS: The one I have is the only thing 
I have looked at. 

Q Which one is that? 

A I have 1631. 

Q Dr. Williams, doyou have any specialized training 
or experience in the field of social services? 

A I have worked in that field and I have taken 
courses in that field. You have to as a part of psychology 
and part of getting advanced degrees. So I would say yes 
from that vantage point. 


Q Do you consider yourself qualified to make 


evaluations of the subject matters of this examination? 
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A I would say on many of the items or the areas 
that are specified here I would feel that I could be com- 


petent to comment in some knowing way. 


Q Did you reach any conclusions concerning this test? 
A Yes. 

Q Would you tell us what they were? 

A I think in general, in the main, it is a good test; 


I had some comments about some of the items. I think 

Dr. Katzell had mentioned some of the characteristics that 

I would also agree with. I think there are some 

specific sorts of things. There was comment about Item 1l 
this morning. I had a thought about that, too. 

In the main, I would say it $& a good examination. 

I happen to have seen, looking over her shoulder, the item 
analysis. As I looked at those tallies I think that supports 
the general view it was reasonably well put together. 

THE COURT: You said you had some thoughts about 
question ll. Do you mean they were prompted by this 
phenomenom of so few people getting it right and more 
people who were in the low group getting it right than those in 
the upper group? 

THE WITNESS: It is related to that. Specifically, 
whenever you get an item -- if one were to run the kind of 


item analysis that would generate a biserial coefficient 
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or another variable of that you would probably be able 

to see the way in which the alternatives are working 
exactly, and if the keyed choice is working in a negative 
fashion you make a certain kind of cecision with respect 
to it. 

Quite specifically we were looking at the choices, 
and I guess someone had mentioned choice A, B or D and 
from a professional vantage point you could argue about 
certain things as to the goodness of the alternatives. 

Q Based upon your observations and your interviews, 
do you have any opinion as to whether the exam in contest 
has content validity? 

A The content validation is partly defined by the 
procedure and the procedure certainly fits the content 
validation. 

Second, from an impressionistic vantage point of look 
ing at anmiinterviewing people who passed the exam and 
knowing where their rank was, what rank that person had on 
the exam, I would say it is in keeping with my impression. 

In brief, the examination is discriminating among 
people on some variable of competence. 

THE COURT: Do I understand you mean when you say 


that there ranking is in keeping with your impression that 


among the people that you interviewed you later found out how 
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they ranked? 

THE WITNESS: I saw it. I happened to see a print 
out. I don't know exactly how, but one of the persons felt z 
had been very close to the top in the test and I confirmed 
that she was. 

THE COURT: How many peoples grades did you see among 
those you had contact with? 

THE WITNESS: I did not see -- I only saw one score of 
an individual, but what I saw as those persons whopassed and 
those who did not. 

Q Did the people in the interviews tell you whether 
they passed or not? 

A Yes, all of them did. 

Q Do you think any other form of validation was 
feasible in this examination? 

A Not technically. 

Q Why not? 

A If one were to use one of the other two validation 
procedures the first being label criterion related validity, 


which would include predictive validity and concurrent 


validty, you would have to have some reasonable large 
cluster of people first in which to derive the criterion 
against which you validate thetest within a select 


set of job functions. 
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Secondly, yov would have to have some stability in 
those areas because it presupposes that you have some reliable 
measure on both the criterion and certainly reliability 
in the test. 

That wouldn't have been possible under these con- 
straints. Construct validity wouldn't have been entirely 
relevant except if one were trying to identify things 
like responsible behavior, leadership,honesty, et cetera, 
and these variables were not being called into question or 
being assessed as being relevant for the test. 

There is nothing that I have seen in the cate- 
gories that have emerged from the test analysis that would 
indicate that you would want to adopt a construct validity 
procedure. 

Accordingly, the remaining one would be content 
validity and that would be the most relevant here. 

Q You testified before you are on a committee which 
wes changing or reviewing the APA standards, and you have 
also been asked to comment on the proposed uniform federal 
guidelines in employee section procedures? 

A Yes. 

Q Is there any change being made in either of 


those two guidelines on the emphasis given content 


validity? 
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A There is a change in the uniform guidelines where thdre 


is much more weight, if youplease, being given to the 
desirability of content validity relative to the other two, 
construct and criterion, the reason being in some instances 
it is not possible in principle to use a criterion related 
validity approach or construct, and accordingly one has 

got to begin to identify procedures in ways that makes 
content validity much more practicable and understood. 

Within the APA guidelines there has always been 
equal weight because the validation procedures are regarded 
as inference schemes where it depends upon the kind of 
inference one is trying to draw as to what kind of 
validation procedure you would use. 

That has been given equal weight within the pro- 
fession, and new APA guidelines there are greater detail 
given as to what might be certain kinds of pitfalls and 
essential steps that must be taken if the examiner or test 
constructor is to use one of those procedures. 

Q I think it was Dr. Katzell who commented about the 
qualification of the test mode used. Do you have any 
comment based on your observation? 

A The test mode, paper and pencil, related to an 
alternative such as T and E or a simulation procedure or what 


have you, is typically a function of some practical con- 
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straints, one, and secondly what kinds of decisions are 
you trying to make. 

The practical constraint is when you have large 
numbers of people you have got to design some way of tr: at- 
ing them all fairly and one an equal basis and hopefully 
at some minimal expense, which is usually thrown in. 

The preference then would be to use paper and pencil 
because, one, you can easily administer to a large sample 
of people. 

Two, it is possible to sample a large variety of 
behaviors that are verbally mediated, written down. And, 
three, it is cheap. Otherwise, you wouldn't be able to 
get through the process of evaluating if you had thousands, 
and some industries do, applicants who were coming through 
for consideration. It is a little difficult I think in this 
instance to think about some other kind of performance mode. 
For example, if you think of people who are primarily agaigned 


the task of reviewing and writing policies for some aspect 


of some sub-title of the agency,*what would you get? ~° Ycu 
would get samples of their written behavior. Some who have 
to go through an evaluator. While other people were making 
decisions with respect to printing and distributing, 

you would have to get samples of behavioral decisions they 


made. I think the most appropriate mode here is probably a 
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paper and pencil test relative to the alternatives that 
I know about. 

Q Mr. Mintz suggested before that the fact that 
there are provisionals serving in a position supervising 
human resources specialist who have received a satisfactory 
rating from their Supervisors might serve as some kind of 
what he called a rough concurrent validity study. Do you 
think that is possible? 

A No. 

Q Why not? 

A There has been ample mention of a halo effect, but 
I think that you don't have the part of the people who were 
judged to be unsuccessful. If they were still there, or 
identifiable some place, and you could get scmne ratings on 
them, that would be fine. But short of that, if there is 
not to be any kind of measure on performance, supervisor's 
ratings or something else which would contain their own 
problems, there would be no way of making the assertion, 
although one might want to, tat the existence of this class 
of employees is prima fucie evidence of validity or of 
satisfactory performance. Unless there is clear evidence 
that every time there is an existence of an unsatisfactory 


act the person is either terminated or there is some mark 


that effect put into a folder there is no systematic 
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collection of data that would permit you to say that you 
have some kind of performance rating. 

MS. LOREN: I have no further questions. 

THE COURT: Dr. Williams, are you going to be 
in New York tonight, are you going back to Princeton? 

THE WITNESS: I live in New York. 

THE COURT: How long do you expect to be? 

Is this the last defense witness? 

MS. OMANSKY: Yes. 

THE COURT: Mr. Mintz, how many hours do you think 
you are going to consume tomorrow? I am trying to see 
if we should go longer tomorrow or if you will have enough 
time. 

MR. MINTZ: As far as I know now, and I don't see 
any reason it would change, tomorrow will include only 
Dr. Williams' cross examination and Dr. Loposes 
direct and cross examination which should take roughly as 
long as perhaps Dr. Katzell. I would presume we could finish 
tomorrow. I am willing to go ahead and cross examine 
now, if the Court wishes. 

THE COURT: If I thought you would conclude your 
cross examination by around 5 o'clock I would go now. 


MR. MINTZ: I can't make that promise. 


THE COURT: Any objection to returning? 


Williams- 


THE WITNESS: No. 


THE COURT: Why don't we start at a quarter of 


tm in the morning? 


We will then have made up everything but ten minutes 


of what we are losing now, and in case we get essed 


we are a little farther ahead. 


(Adjourned to Friday, January 11, 1974, at 


9:45 a.m.) 
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James C. Jones, et al. 
Vv. 
73 Civ. 3815 
The New York City Human 


Resources Administration, 
et al, etc. 


New York, January 11, 1974 
10:15 a.m. 


(Trial resumed.) 


THE COURT: Good morning, ladies and gentlemen. 

My apologies for keeping you waiting, but the 
weather was a little too much for me out my way. There was 
ice all over. 

I think we are ready for the cross examination 
of Dr. Williams. 

EVE S39 B. WILLIAM 8S, resumed. 
CROSS EXAMINATION 
BY MR. MINTZ: 

Q Dr. Williams, would it be correct to characterize 
your experience in psychology, perhaps your training.and 
experience, as primarily clinical rather than the field 
that is generally known as industrial psychology? 


A No. 


Q It would not? 
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A No. 

Q In your explanation of your background my recol- 
lection is that you mentioned work with ETS, which is a 
testing company. I don't recall anything else of that 
nature. Could you refresh me on other industrial 
psychology work you have done? 

A Yes. My formal specialty in graduate school was 
in the clinical program. In these clinical programs 
you have a wide variety of latitude with respect to some 
particular areas. 

In my particular case I was involved in both 
test instruction, as a graduate student, for the purpose 
of diagnostic testing; and, secondly, for the purpose of 
trying to establish predicted measures that would describe 
the behaviors of drivers. 

This effort occurred prior to my receiving the 
Ph.D., and years after that. 


And after that most of my professional behavior 


was involved with organizations such as the American 
Occupational Therapy Association, in which I was a consultant 
for the certification exam for eight years, and involved 
evaluation of teacher corps training, and the kinds of 
objectives that were set in educational systems were those 


particuiar programs, as well as the evaluation of the kinds 
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of teaching context in which Peace Corps volunteers would 
find themselves, and the evalustion of those volunteers. 
So in the traditional sense of clinical practice 
doing diagnostic testing with projective devices, in- 
dividually adminstered IQ tests, that was a part Bf 2; 
but that was not by any means the major part. 
Q Well, wat is the difference between testing for 
diagnostic purposes and testing for employment purposes? 
A Well, if you think of diagnostic testing within 
the context of a clinic or a hospital, typically this 
occurs in an individually administered testing session 
where the psychologist or the psychometrician might use 
a battery of tests to gain some kind of assessment of 
personality funct.on, aptitude or IQ level. 
If he is called upon to give some kind of opinion 


with respect to appropriate treatment patterns, he would 


test for that, or the presence or absence of some organicity, 


brain functioning, and so on, he would do that. 
On the other hand, in many instances,not all of 


them, in many instances of industrial personnel testing, 


one tends to use a group setting as opposed to an individual 


setting. One does not tend to use some of the kinds 


of personality devices that are found in clinical settings, 


Such as the Rorshak, TAT, although one can use those at 
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certain levels, and I am not so sure that there is a sharp 
line. It depends upon what level. 

For example, in industrial psychology, if one is 
trying to appraise a person for an executive position, 
typically that is done with an individually administered 
kind of battery. 

Q We are dealing here with an examination designed 
for and administered to something in the neighborhood of 
11 or 1200 people, more or less, simultaneously. 

Is that a different experience than the clinical 
diagnostic testing? 

A That is a different procedure. That is more 
comparable to what happens or what is done for a law school 
or SATs, where you have got a large group of people where 


you have got to administer tests under some stundard con- 


dition. 
Q Does your role in ETS involve test construction? 
A Test development is the way we refer to it, and 


I only add that qualification for the following reason: 
The test developing activity occurs with a whole team of 
people, with content specialists from the outside, with 
experts from the inside, and most of my involvement is 
very minimal, although there are some, esrecially with 


the --there is some involvement that I have had with a 
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National Teacher examination, which is one examination. 
But in the main, no. 

Q Then you don't in your experience write test items? 

A No. The writing of test items is done at ETS by 
outside committees. 

Q Have you ever conducted a job analysis personally? 

A Yes, I have. 

Q When and where did that take place? 

A That took place while I was in New York @-= 
Columbia as a research assistan*. and also as a research 
associate in which we were doing two things: 

One, we had to try to understand what were ‘he 
real life context situations in which truck drivers, 
especially Esso at that time, now Exxon, truck drivers 
found themselves with:n the densely populated area like 
New York City, or « not so densely populated area like 
Long Island, or other areas around the country, such as 
Chicago, which is one. 

Second, we had to do a jot analysis, but we used 
a different technique, namely, that of a critical incident 
technique, in which you invite other professionals or experts 
to define an instance of very good driving behavior 


and an instance of very bad driving behavior, according 


to their experience under different conditions, such as 
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daytime driving, nighttime driving, et cetera. 

The reason we use that is that there is a 
difficulty in trying to observemost of the significant 
instances of driving, so you had to depend upon others 
who are in that context more often and knew different 
aspects of it to define those incidents. 

After we did that the whole categorization process 
continued in the office, the item writing, the elimination 
of items the review of items, the construction of pilot 


tests and reconstruction, and so on. 
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Q You state you were involved with this entire 
vroject? 
A Yes. That was a threeeyear grant we had. 
9 Is there a distinction between designing a 


test to examine a particular construct such as driving 
ability as compared with the test to examine the entire 
range Of abilities necessary to perform a job? 

~ 


f, 


The process isn't significantly different. 
The difficulty is different, where you don't have tasks 
that are well defined you have got to make what are usually 
called second order or third order inferences about what 
underlies what you see and then have some kind of con- 
firmation of that, either through content validation 
with a colleague or some substantiation process. It 
is difficult that the process is not, t!ie logic is not -- 

o It is more difficult to do the order? 

A It is difficult. Any time you don't have a well- 
defired set of tasks you have a greater difficulty. 
You are setting parameters on whatever you are doing 

Q Are there any other occasions on the one a 
involving truck drivers in writing job analyses? 

A For professionals such as computer programers 


and key punch operators and unit record machine operators, 


which one of the responsibilities that I had, not for the 
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purpose of test construction, but for the purpose of per- 
sonnel appraisal by the personnel office, writing job 
descriptions for computer proqramers, key punch operators 
and unit record machine operators, 

Q Is there a difference in the nature of the job 
description when you have that difference in purpose? 

A NO. No, the only -- I suppose if one were 


to cite a difference it might be somewhat in the 


specificity of language, because there are times as admini- 


strator you might put certain things in a particular way 
because you know it is going to be acted upon by another 
office, If you have an expectation that someone 

has got to go through the behaviors of constructing a 
test item you might want to make it a bit more specific 
in some instances in order that the next person has some 
Clarity as to what is intended, 

Q Other than the test for driving ability, have 
you ever been involved with designing == and you also 
mentioned one for occupational therapists? 

A Yes, 

9 ~~ have you ever been involved with designing 
an cmployment examination? 

A Yes. Well, that is a certification examina- 


tion which leads to employment, No, I haven't been 


18 


19 


xr & 8 B 


jwa3 Williams-cross 520 
involved in other projects aside from those two, 

I have assisted in the design of tests for secre 
taries, a content validat’ on test, I call it a small, 
informal effort because we were assisting the personnel 
office on their testing program, 

Q My experience may or may not square with 
vours, but it seems to me that the classic example of the 
content validation test is usually the secretary's test 
because if you want to find out if someone can type you 
have them sit down at the typewriter and see if they 
can type. Isn't that one of the easiest things to 
do? 

A The fact is in the university setting you have 
some offices that would require a secretary to type a 
lot of statistical tables and some don't type that very 
well, because they have to interact with the carriage 
in ways some don't want. 

Others might request a secretarv and want 
typing skills of 40, 50, 60 or whatever words per minute. 
The fact is she doesn't spend that much time doing 
that, she might spend time answering the phone or filing 
papers. 


There is a'tways a need to be sure that the 


weight given to typing was not extreme, But if it is 
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just straight typing in the sense I think you might have 
meant it is fairly straightforward. 

Q Do I glean from what you just said that it is 
important to be very clear that you know what the 
person is actually going to do and not to assume he is 
going to have a generalized test? 

A I think in that context that I referred to, 

I mentioned before I was in charge of the affirmative 
action program at Teachers College and we were con- 
cerned that various applicants, especially minority 
group applicants, not be rejected on an irrelevant skill 
if in fact it is not going to be called upon. You have 
to look and see if that iswhat is being called upon. 

Q You mentioned yesterday what you referred to 
as seven steps in content validity. Unfortunately, 
the transcript isn't in and my note-taking wasn’t that 
fast. Would you repeat those, please? 

A I should hav: said six rather than seven. 

Yes, one is to conduct the task analysis 
or field audit as referred to here. 

The second is to draw some inferences about 
those observations, to identify areas of functioning 
which would then be used to define the items, so on. 


The third step is to decide on the format, you 


co 


jwas Williams-cross 522 


know, the test, and one might have at least four options; 
one could have a written exam, an oral exam, a simulated 
situation or a replication of some sort. 

The fourth is, once you decide that, to then 
go forth with the item construction which would permit 
you to sample the domain of skills or the knowledge 
tnat you are looking for, 

The fifth is to assure that you have some 

idard administration procedure such that candidates 
are not penalized because of variations in the test- 
taking diniihintats 

And the sixth would be some means of 
estimating the reliability of the components or areas of 
the test, and there are different strategies for that. 

This last one becomes a little bit problematic 
where even in content validation procedure it is not al- 
ways the case that a reliability estimate is either 
possible or desirable, hut in other cases that it is 
there is a question of what kind of estimate would you 
use. And some of the standard ones -=- or I should not 
say standard ones, but some that are more well known 
would not be in various contexts that would cualify 
it. 


Q I am not certain what you mean by the second 
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one following the task analysis where you draw infer- 
ences and identify areas of function. Could you 
explain what that entails? 

A Sure. For example, if I were to concern 
myself with doing a task analysis for a typist and 
observed that their finger movements, their fingers 
touch the keyboard in certain ways, I could therefore 
draw an inference or abstract, if you please, and say 
some motor facility is involved, There has got to be 
some way of estimating hos agile this person is digitally 
or to what extent is there a motor-visual coordination. 

The motor=visual coordination would come 
into play if one has got to read some text in one place 


and do something else in a different place. 


THE COURT: Like erasing a typo? 
THE WITNESS: Advertently or inadvertently, 
yes. 
A Then one would say there is a perception- 


motor skill that is involved and I might want to have some 
estimates of this perception=motor skill in different 
wayS. 

You then draw some inferences about the nature 


of the perception-motor skill. Is it one that requires 


a great deal of content of control first before the 


_ 
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person can develop it, or is it one that you can do by 
just rote practice, that is, more time on the machine, 
the greater the possibility of speed of the person will 
increase? Or is it the kind of thing where you 

have got to think about some instructional sequence that 
will help them in the control of the visual-motor 
project, That is one, 


Another is «= 


THE COURT: Doctor, I hove that Mr. Mintz 
feels it is an adequate answer, I feel it is an 
adequate answer. We have to move along. 

“ARe MINTZ: Yes. 


BY HR. NINTZs 


Q Where do you see this particular step as 
having taken place in the developmoat of the exams at 
issue here? 
A As I sit here obviously what I was talking about 


was an inferential process that the job analyzer has to 


go through, You don't have to see it, but he, through 
inferences that evolve, the tasks that were observed, 

ne observed or learned through other people and then tried 
to summarize these observations into a smaller set of 
descriptors of the sort that appeared on that —- 


THE COURT: Cardboard, the test plan? 
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THE WITNESS: Right, the test plan. 

A I don't ses but I assume it is probably the 
case that he had notes about these things too, but I 
couldn't ascertain that, I have not seen that. 
But it is that step of going from some observation to a 
verbal description which is understandable by some set 
of people who will be involved in the act of putting 
together the items. 

Now, the mental processyou don't see, Lut the 
result of that process is put in the set of the descrip- 
tors of the test plan. 

Q You made some assumptions about a set of 
notes you haven't scen, 
A I take that back. 
THE COURT: You don't need a set of notes. 


You can make inferences without notes? 


THE WITNESS: Sure. 


THE COURT: Gentlemen, I am concerned here. 
I have listened for about two days with interest, but 
nevertheless for quite a while, to the general principles 
of validation and s0 on. JT would really like to 
get down to brass tacks. 

I recognize you have a perfect right, of 


course, to try to bring out the limitation of Dr. Williams‘ 
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experience, but we have to get along. We all have 
limitations in our experience but now I would like to get 
down to the substance of it, 
Mk. MINTZ: That is what I am trying to do 

at this point. 
BY MR. MINTZ: 

Qo I don’t know if you have any material w'th you. 
I am referring to Nefendants' Exhibit F, the officially 
denominated field audit report, dated February 4¢h. You 
are familiar with that? 

A Yes. 

fa) Do you know where “he items listed at the 
bottom of page 2, numbered 1 through 8 -= those eight 
characteristics -- do you know where they came from? 

A My assumption is they are based upon the observa- 


tions that lr. Rosenberg made of what people are doing, 


as well as learning them from their supervisors what 


they were doing, and then he drew an inference that 


. needed the knowledge,‘ i'r example, or functions that re- 


lated to public and private institutions, 
Q This is an assumption on your part? 
A Right. There is no other way. You. 


have got to draw an inference about things you see in this 


context. Assuming that he is a rational man, that is 
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2 the only thing he could do, 

3 Q Is it possible that these eight items or eight 

4 Or seven or nine, perhaps, very similar items existed 

5 prior to even October 1, 1971, when he did this job descrip 
6 tion, and that in fact he later Plugged in -- 

7 THE COURT: I will sustain the objection 

° to that. Of course it is possible, but Mr. Williams’ 
view about what is possible or not doesn't help me, 

10 Let's get a Rosenberg back here if you want to find 

Il that out. 

2 MR. MINTZ: My point is it would affect 

sad the quality of job descripi ons, 

" THE COURT: If you want to ask if it were 

Ib not based on lr. Rosenberg's experience and inferences - 
6 would it affect the4ob description, I think that is a i 
" good question, 

” BY MR, MINTZ: 

19 Q If these eight behavior descriptions or skills | 
a aid knowledge that they referred to in fact preexisted 

21 in doing a job description and they were simply plugged 

2 in, they were preconceptions, would that affect the 

s quality of it? 

aA A Yen would have reason to believe in the quality 
se of it then. 
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THE COURT: I may say, gentlemen, that judges 
are crecited with a certain amount of common sense and 
I think any judge would realize that if somebidy had 
preconceptions about what a job was he would just put it 
down on paper without reference to what he actually 
observed and that itself would undercut oes whole proposi- 
tion. 

MR, MINTZ 3 I think that is critically 
ieepiidte sir. 

Q You stated in this case that you interviewed 
eight -- pardon me; I believe it wae 12 people at Humas 
Resources Administration, of whom I believe you said 
eight were then functioning as supervising human 
resources specialists, is that correct? 

A I am not really sure of all their titles, but 
X would may that is ecoveutmaneby correct, and I 
yesterday gave the reporter my list and i don't have thei: 
names before me. He was jus¢ going to give them back 
to me, | 

Q Of those approximately eight who were functioning 
as supervising HRSs, ata they include both people who 
had passed and people who had failed the examina.) * 


A They did. 


Q Did I understanc you to say that you dxe«w some 
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correlation »atween your observation uf them and their 
testifying? 

A My clinical impression was that the people who 
passed the test were somewhat different with respect 
to their -- certainly their verbal facilities, certainly 
their knowledgeability about the agencies in which thev 
were working than those that didn't, That is a 
clinical impression, 

Q Are you seriously contending here tiat on that 
basis you could make a judgment as to the relative 
capabilities of these eight people that performed their 
jobs? - 

A NO. 

Q S0, in other vords, it may well be those who 
had less -- {I don’t know how you call it, but thellower | 


clinical impression could in fact be the best people to | 


do the job? 
| 
A Quite possibile, 
Q You also stated, as I recall, that es you under- 


stand the situation here, comparative validity -- pardon me; 
any sort of sriterion validity test would not be feasible? 
A Yes, it would not be technically feasible. 
Q You are aware, I presume, Dr. Williams, 


that a very substantial number, well over a hundred, 
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we 


perhaps in the nature of 150 or more ofthe people who took 
the test, were then and there at the time they took it 
functioning as human resources specialists, senior 
supervising human resources specialists? 

B I am not aware of what they were function as. 
X am aware they were there, I do know, or did know, 
that they were there, The particular titles they might 


have had I am not sure of, You were saying that 


they were senior. 


—" 
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0 I meant supervising human resources specialists 


by provisional appointment. 


A They were senior HRSs, and I don't know that. 
Maybe they were, I am not disputing it. I just don't 
know. 

Q Their permanent title is not relevant, The 


point is they were functioning in the job for which 


they were taking a test, and there were something like ~- 


THE COURT: The test, right? 
MR, MINTZ: The test. 
QO There were something like 180 people functioning 


in that job and the evidence here is that a substantial 


number of those 180 people did take one or both of 


these tests. Are you aware of that? 
A Yes, I am aware of that. 
Q Wasn't that in fact a rather uniquely well- 


presented situation for performing a criterion-related 
validity test in that performance valuations of these 
people could have been taken at aboutthe time they took 
the test and compared with their test results? 


A There are two things, One, indeed, if there 


were a system of performance evaluation in effect 
that gave you that kind of data,that would certainly 


be a major assist. The other problem in the criterion- 
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related procedure, especially in predictive validity, is 


the number of cases you have -- if one were thinking about 
doing it for each specialty one would lave a problem with - 


the number of cases that would be found in some given func- 


tional area. 


It is possible to have a rating. You would 


have to go through procedures that would get you around 


the supervisor halo effect or the supervisor bias, 


and the bias has some various interactions of the 


same ethnic qroup and a different ethnic group rating 


each other. Theoretically, all of these are possible, 


but under the circumstances my understanding of what exist 


in both the agency and what the personnel office was 


asked to do, they weren't asked to do that. 


THE COURT: 


You mean they just didn’t have 


the system of joing that? 


THE WITNESS: 


THE COURT: 


Yere 


That is a separate question from 


what Mr. Mintz raises, though it is influenced by it, 


and raybe the explanation of why it wasn't done, but in 


your opinion would it have been an opportunity to perform 


a criterion validation if in advance the rating structure 


system had been established. 


THE WITNESS: 


If you have all that system 


a 
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in place and you got around all the other pyschometric 
problems of reality and the validity of the criteria -- 

THE COURT: I don*t know why all those prob- 
lems would be different in this situation than in any 
place in which you are measuring performance of people 
on the job. 

THE WITNESS: If I may, there was a recent 
study which culminated six years of effort at ETS trying 
to evaluate tests against six occupational categories. 

In using supervisors’ ratings you have 
interaction, There is the black supervisor rating, the 
black supervisee, the black supervisor with a white 
supervisee, and all the combinations thereof. The 
results are quite clear in terms of the functioning 
biases, Accordingly, one has to go into a more — they 
are sometimes referred to as psychophysical methods of 
using triads or constant stimuli or some approach s0 
labeled, 

THE COU™.T:s I understand what you are saying, 
Doctor. All that I am asking is whether what you are 
talking about, the problems that you are talking 
about, if you are going to rate people on the job, 


aren't simply an inherent risk always of rating people 


on the job. 
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THE WITNESS: Yes. 

THE COURT: And at least the social sciences 
as you have described it have developed an awareness of 
this possible distortion and some capability, I imagine, 
of discounting it or correcting it. 

THE WITNESS: Hopefully. 

TUE COURT: Again, I proceed on the assump- 
tion which you and your colleagues have made clear, 
that your skills, however great they are, don't produce 
perfection, and I don*t think anybody expects them to. 

Excuse me, Mr. Mintz. Go ahead, 

BY MR. MINTZ: 

Q Are you also aware, [Dr, Williams, a sub- 
stantial number of people who took the test are still 
functioning as supervising human resources specialists? 

h I don't know that, but I assume it is the case. 

Q Again, with the limitations you have expressed 
about the general difficulty which e« ists in any perform- 
ance-rating operation in the period since this suit was 
filed from September until today, would it have been pos- 
sible to get performance ratings and get rest resulta? 

MS. OMANSKY: Objection. 

THE COURT: Overruled. If you can answer 


it. 
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MS. OMANSKYs I think even as an abstract 
matter it is unfair to ask Dr. Williams that, because he 
is not aware of the injunction against doing anything with 
provisional employees, and I think the context, unless 
it is totally in the abstract -- 

THE COURT? Let me hear the question, 

(Question read.) 

THE COURT: I will sustain the objection 
because I don't quite understand it. 

MRe MINTS: Dr. Williams said yesterdy and 
reiterated today a criterion-validity-related study 
is not relevant in this context. He agreed it would 


have been possible in October of 1972, when the exan was 


given, to get concurrent performance ratings and compare 
them with test results. I am asking him similarly 


would it have been ponsible -= 


THE COURT: I wouldn't say he as agreed 
if there had been an evaluation system it night have “ 
been useful to do <o Of course, there wasn't any 
a 
evaluation system as I understand, 
MRe MINTZ: That we know. 
MS. OMANSKYs First, I am not sure he 


agreed it was possible. 


Secondly, I don’t inderstand the deduction 
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from the abstract to this particular case. That is an 
inference. 

THE COURTs Mr. Mintz hadn't finished his 
explanation, 

Yes, Dr. Williams has in my estimation -- 
if I am wrong about what you intended to say, please let 
me know -= agreed if there had been an evaluatim system 
in effect at the t’»e it would have been useful, I will 
use that word, as a criterion method of validation, to 
use that evaluationsystem, pointing out the hazards 
that existed, which I indicated I recognize. 

Now, what is the next question you wanted 
to ask? 

MR. MINTZ: That would have permitted the 
development of what would be known as a concurrent validity 
study, that is, comparing present performance with 
present test results. 

THE COURT? Right. 

MR. MINTZ: What I am asking now is if it 
would also have been possible to do a predictive study. 
You have test results of a year ago, performance as of 
NOW. Could you have gotten performance ratings and 
compared those two criteria? 


THE COURT? I see no objection to that ques- 
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tion, If there is some premise that you think is 
implied tiat Dr. Williams doesn't know about, he should 
be informed of it, When you talk about the injunction 
I assume you mean the practice of the Civil Service 
Commission, 

MS. OMANSKY: I think the premise vas if in 
the interval between September and now the Department 
of Personnel had gone out and did rating studies, and 
obviously if the Department of Personnel -—~ it would 


have been difficult under the tenor of the injunction, 


THE COURT: You mean the court's injunc- 
tion? 

MS. OMANSKY: Yes, 

THE COURT: You are free to argue that. 


MS. OMANSKY: I think the question as it 
was rephrased might be less objectionable, if that could 


be repeated. 


THe COURT: All right. Then if you will 


read back the rephrased question, Mr. Reporter, 


ae 


(Question read.) 


A If you waive the policy restrictions you could 
do all those things. f 
Q Is it normally the case that the people who 


are taking an exam for aparticular position are then 
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and there performing the functions of that position? 


A I am sorry, I 1isse! a word, Would you 
repeat it, 
9 Is it normally the case that people taking an 


examination for appointment to a particular position are 


in fact then and there performing the functions of that as 
position? 

A Is it normally the case? 

Q Yes. 

aA It varies. It depends on how they are or=- 
ganized. There are many instances, because of a 


shortage of personnel or manpower, people are put into 


a role where they have to have some kind of certify- 


ing examination. That occurs cuite often. iven pro- 
fessionally that occurs. You could have people function 
as a lawyer without having passed the bar exam. There 


are some critical functions they could perform without 


passing the bar exam. 

Q When vou have such a circumstance existing, 
doesn't that give you an opportunity, if you care to 
take the time and make the effort, to in fact do a better 
job of evaluating your test? 


A You a)-vays have an opportunity. My response 


to the task that you are asking me is that under the 
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existing constraints what can you do? If one had un- 
limited finances and unlimited resources and no 
restrictions with “aspect to the rvles and policies, I 
am not so sure there isn't anything t!.at you couldn't 
ado -- well, uot too many things you couldn't do. 

THE COURT? What restraints are you talking 
about about anredictive validation? Once the test was 
behind everybody I am not clear what restraints you are 
referring to. Before it was taken I understand there 
were reasons not to speak perhaps to those who might 
take the test or alert them as to what might be the 
questions, I can see that might relate to an evalua- 
tion on the job beforehand even, but afterward what 
restraints would there be? 

THE WITNESS: If I understand Mr. Mintz 
correctly he shifted from conc. ‘rent to predictive 
validity, In this situation one would want to have a 
reliable criterion which may he other than supervisors’ 
ratings but performance output -- 

THE COURT: There must be some way to 
find out whether people are doing a good job or not. 


Sometimes I feel like I am in a never-never land when 


I talk about this, kre the problems here any different 


than they are for ail of us in our everyday lives? 
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THE WITNESS: Not if one defines carefully 
what one expects from HRA, sup or senior. 

THE COURT: Do you suggest thet is harder 
to do in the class of employees we are talking about 
here than in other instances? ai 

THE WITNESS: Yes, I do, because if you 
talk about a person on ar assembly line in an automobile 
plant you can count the number of rivets and bolts he 
puts in, It is not easy to count the units of out- 
put in a social science -- in a social service context, 
60 you are never sure. For example, one can count the 
number of lines of code that a person writes, but one 
doesn't necessarily know whether it is good code. One 


can count in a policy-making position the number of 


sentences one writes -- 
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Q Then are you saying, Dr. Williams, that in this 


setting, you don't think, regardless of policy restraints - 


forget policy restraints - that a predictive evaluation 
couldn't effectively be male because your profession has not 
yet developed to a point where it could rate the job performance 
of the employees here with sufficient standardization to 
be able to compare it with the test results, or are you 
just saying that it is tough to do that anywiwre? 
A It. is tough to do that anywhere. T wouldn't 
argue that it is not possible in principle. 
As I saw these positions and the people in them, 
it is not always clear what a unit of work is. One could 
take cevtain common sensical kind of measures, such as, 
does a person appear on the job regularly or on time regularly? 
One could get some ratings from the comnunity, 
if he is a community worker - does the community like this 
person or not? 
Well, if likeability is a success indicator, 
that is one thing, but if being liked and still being able 


to move the community along, making it more self-sufficient, 


are taken into account, that might not be sufficient. 
That is to say that being 1’. able is not “he same thing 
as helping the community mobilize its own resources to do s 


thing about whatever problems they he So it is hard to k 
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eat it-is. 
Q Well, I can understand the problems involved there, 


but don't those things relate, once we get away from 
assembly line workers or typists who do nothing but type, 
the large millions of service jobs that we have in the 
world today, don't your comments in general apply to service 
jobs? 

A Yes, they do. 

Q And doesn't that add up to the fact -- and I don't 
say this is bad; I am just trying to get to the point -- 
that there are substantial difficu'ties in the job measure- 


ment part of predictive validity? 


A That is correct. 
Q The job performance measurement part? 
A Yes. 


BY MR. MINT2Z: 

‘2 Don't the same factors which make it tough to effective- 
ly rate »oriormance in 1 complex jo» such as this make it 
tough to design an exam which will effectively evaluate and 
predict performance in a job such as this? 

A There is a compound factor here, but let me firat 
say no, and I will use an example. 


For example, we are presently involved in validation 


studies using the LSAT and the multistate bar exam, there 


» 2 8 8 


—OOoOoOoOoOoOoooooo OO EEE EEeeeeeeeeeeee——e—e—e——e—e=e=eee=SaeEOSeeeeeeSeeeee 


543 
slbr 3 Williams~cross 


was one criterion or set of criteria that are reasonably 
firm, such as schools and grades in law school, first, second 
and siird year. 

On the other hand, none of this can be shown to have 
any relationship to the effective performance of what a perso 
does after all that. 

However, it still makes sense to say that I would 
like to insist upon a certain base level of knowledge and 
skills for the person to enter the practice of law. 

Now, when you ask the question, is it more difficult, 
it depends on what kind of inferenceyou are trying to draw. 
If one is trying to draw aninference which would permit you 
to predict from the test to some event in the future, Like 
predictive validity, that is terribly difficult. 

If one is trying to draw an inference from the test 
and say "I think this person has a command over this set of | 
skills and knowledge,” that is not difficult; that is the 
content validation strategy. 

Q That study you refer to involves, in fact, an 
attempt to develop correlation between LSAT scores and 
law scheol grades and bar exams performance, is that not 
correct? : 

A Those are criteria that have been e:.umerated. 


However, there is an interest in trying to identify what 
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Q But that study does not do that, does it; it 


is an effective lawyer. 


does not suggest that it is going to evaluate lawyers 
performance and then compare that with -- 

A All the specifications for the study have not been 
completed by the law school council, so it is not clear 
what direction they might want to take, but I just happen 
to know there is an interest in being able to make that 


kind of thing. 


Q Have you seen the specifications up to this 
point? 
A Some, yes. 


Q In fact, no one knows with any real certainty cial 
LSAT or law school grades or bar examination results predict 
effective performance of a lawyer? 

A That is correct, because there is not a clear 
definition of what effective performance is. 

Q Isn't the same prceblem involved here. that no one 
knows whether the performance of this exam  ..: stively 
predicts performance on the job of superving human resources 
specialist? 

MS. OMANSKY: Objection. There is a premise that 
thia examination was supposed to predict performance. 


THE COURT: I am not proceeding on that premise, 
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but I will allow the question. 
A Sure. 

THE COURT: I think with regard to all validation 
endeavors, it is not a question that the law necessarily 
requires validation, but that validation, to the extent that 
it may exist, may prove whether an examination is legally 
sound or isn't legally sound, and I think Mr. Mintz is 
entitled to ask what the problems of validation are in this 
situation. 


Did you understand the question, Dr. Williams? 


THE COURT: Well, you didn't have the same 
problems of trying to predictively validate, as I under- 
stand it, with regard to the jobs such as we are dealing 
with here that you have with regard to performance as a 
lawyer? 

THE WITNESS: Yes, I was about to say that, that 
there is no claim for this test that it is predictively 


valid. 
BY MR. MINTZ: 

Q Would you say that the purpose of a test such as 
this is to predict as well as one can the performance of 


people as supervising human resources specialist? 
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A Not as I understood it, 
THE COURT: Overruled. 


What was the answer? 


A Not as I understood it. 
Q What is the purpose of the test, then? 
A As I understood it, there is an administrative 


problem that the city has of trying to identify a setof 
persons to be put on a list from which you could draw them 
at some future date according to their rank to fill certain 
slots that are available. 

The second is to ascertain whether there existed 
a minimum set of knowledge and skills among these persons 
which would be expected on the jobs that they might be 
called upon to perform, 

There is no assertion of predictable validity. 
All it is saying is that ore can certify that this bit 
of knowledge exists. This is the same way as saying that 
you .% 12 certify that a person knows how to add and 
subtrect, cat that does not say he will be a good accountant 
or anything e. +4. 

Q When yo: refer to a minimum level of knowledge and 

skills, are you able to come to any conclusion about, in 
fact, what level, as it relates to performance on this 


exam, is the minimum that one could have to effectively 
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perform this job? 

A As I talked with the various people who were doing 
widely different things, it did seem to me tat the task 
that some had been called upon to do required, for example, 
in some instances very good reading skills. It certainly 
required some feel of dealing with people voth within the € 
work context and outside of it, as they cite examples if 
problems that they ne~? been confronted with -- | 

Q Excuse me, Doctor, but perhaps you di:t un /¢rstand 
my question. I am not referring to the whole runge of 
the job; I am referring to whether you are able t say 
what minimum level of performance on this test -- 

a You mean like a score? 

Q Yes, exactly. 

A No. 

Q Finally, I believe you said yesterday -- 


and again, regrettably, we haven't gotten the transcript 


supervision abilities. 
But do you recall speaking on that point? 
A I think in the context of construct validation, 
yes. 


yet ~- something regarding .ne testing for leadership and 
Q What was it you said about that? 


A I think I was illustrating that if one wanted to 
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substantiate the presence or absence of an attribute or 

a psychological characteristic such as leadership, respon- 
sibility, or honesty, or dominance, one would use a 
construct validation procedure. 

Q Not a content validation procedure? 

A No. The only exception is that within the con- 
struct validation procedure it is inclusive of other pro- 
cedures such as content, predictive, everything; so it is 
possible one could include -- they are not mutually exclus- 
ive in the sense that you wouldn’t use something or be 
a part of one or the other when you're trying to affect 
the other. 

Q Do you have an opinion, Dr. Williams, as to 
whether good performance on this examination would be 
indicative of any level of supervisory abiolity? 

A It is indicative of whether they have possession 
of certain mowledge and skills abouc the agency. 

Q About the agency? 

A Yes, about HRA and the related content that has 
been defined -- 

Q Do you recall that approximately the first 25 
questions on that examination don't mention HRA or anything | 
to do with it, but simply ask about general principles of 


supervision anda the like? 


~ 
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A Yes, I do. 


Q Can you say with any degree of certainty that 
someone who answers most of those questions correctly 
will be a better supervisor than someone who answers only 
a few of them correctly? 

A I can't say that. All I can say is if a person 
knows or doesn't know it. 

THE COURT: I don't know whether you are in a 
position to answer this, but do you consider it a 
reasonable assumption that somebody who bnows correctly 
all the answers to the first 25 questions may, »r is likely 
to be a better supervisor than somebody who doesn't know 
all those answers? I mean, is knowledge irrelevant 
to the job? 

A I would just personally say knowledge is, but the 
reason I stich very closely to my notion of content vali- 
dation -- well, if I may have the indulgence of the Court 
for a moment, within the context of teaching one can find 
teachers who know a great deal about the subject matter 
who are not good teachers, and there are others who are 
excellent teachers who do not come up quite so high in that 
knowledge domain; so you can't draw tha inference, although 


it seems quite reasonable if you want a teacher to teach 


2 
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algebra in a high school, that person has to know something 
about algebra or calculus, and he or she either knows it 

or he or she doesn't know it. 

Now the other problem is, how does he or she get 
it acro- to the students, and how do you clarify the 
understanding of those students, which is a different 
problem altogether, so you just can't draw the inference. 
You would like to, and maybe in the vast majority of 
cases, that would hold true, but, unfortunately, there are 
many times when it just is not true. 

Q Finally, Dr. Williams, are you aware that the 
people who took one or the other of these two exams in issue 
may have had anything from a high school diploma or high 


school equivalency up to graduate degrees in one or another 


field of relevance? 


A I understood that they had a varie.’ educational 


background. 


Q Would you be able to form any opinion as to the 


relationship between formal education and performance on this 


examination, whether there would be any? 

A I could hazard one, but if one were to ask me what 
evidence I base it on, it would be more on experience, 
in terms of the date at hand, and I would say probably 


that persons with a greater amount of formal education 
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to 


would do better than those with a lesser amount of formal 


: education. 
. However, it is not always the case. 
. Q Doctor, you indicated yesterday, again according 
. to my recollection, thx’. this was a fair test, is that cor- 
: rect? 
. A Yes. 
; Q Are you aware of the fact in evidence here that 
_ the scores assigned to blacks and Hispanics on the average 
| a were substantially lower, the passing ~rades of blacks and 
” Hispanics were substantially lower than that of whites? 
” A No. 
” Q Do you have any explanation for that ‘or any comment 
* on that? 
¥ A I empirically don't know. I have conjectures about 
" it, yes. 
= Q Did knowledge of that in any way affect your 
| " view that this was a fair test? 
” A No, it does not, unfortunately. 
” THE COURT: I would suppose that is what the 
" doctor's testimony is all about. It means an experienced 
= person who has been very much intereste! in bringing about 
» the very objectives that the plaintiffs are interested in 
' ™ bringing about - it doesn't mean he is right or wrong, but | 
| 
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cer tainly he must have thought about whether what has happene 


here renders his opinions questionable in his own mind. 
Am I right, Doctor? 
THE WITNESS: I have conjectures about it; I -- 


THE COURT: I am not asking about accounting for 


Ge I am asking whether I am correct in assuming that 
you have searched your own conscience -- if that is the word 
to use -- as to whether your estimate of fairness can stand 


up in the light of the fact that this has had the unhappy 
result that nobody in this room wants it to have had. 

THE WITNESS: Yes. 
BY MR. MINTZ: 

Q On the basis of your experience, do you think, and 
even given the strictures of the Civil Servj e System that we 
are working under, do you think an exam with less discrimina- 
tory impact could have been devised here? 

THE COURT: That is, on which the blacks and 
Hispanics did better in comparison with whites? 
MR. MINTZ: In comparison with whites, yes. 

A And still have the characteristic of heing job 
related? 

Q Yes. 


A That is difficult to say. I am inclined to say 


I doubt it -- there are sone problems in this situation that 


N 
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are really unique -- but I doubt it ir this instance. 
I could imagine it in some other instance -- 

THE COURT: Ma’ i «sk you this: 

If the examination nad been broken into parts in 
relation to the various sub speciaities, or whatever you 
want to ~all it here, do you think that that might have 
produced the result, with the fund of knowledge being 
tested about not being so broad, the minority group appli- 
cants -- or the test might not have had the pattern result 


it had, or is that all a specula:‘ive area in your opinion? 


A I would think it is conceivable. This is one type 


of problen. I think the assumption behind the generic 
title is ability across areas of functions. 

On the other hand, if there existed a stable area 
where that certainly was not the expectaticn, and one 


might have preferred to have a test that just focused cn 


that, where one was thinking about, say, a financial auditing 


test, one might just construct a test for that. 

But, if I may, I think the problem here is larger 
than the test. I am not oatee the test might not have 
contributed in part; I don't think it contributed much to 


this problem, and I think that if one looks at the sources 


of possible variance you can look at the candidates themselves 


with res,wct to either the assumed abilities that the 
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test is getting at, or knowledge and skills that the test 

is getting at, and/or the attitudes which usually are ignored, 
but attitudes which people have toward tests and their 
response to it, and I certainly think, as we have been 
deliberating here, that the characteristics of the test, that 
is always a place to look, but I think there are some 
assumptions, programmatic assumptions that might have existed 
in the agency in terms of how much experienc’ would overcome 
some kind of deficit, because as I have listened to people 
talk about this, the assumpt was that if brought in under 
the usual civil service route, many weuld not have passed. 
Then I assume there would have been an assumption that 

given certain types of experience one co.ldhave passed anoth2 
kind of test at a later time. It could very well be, but 
programmatically that assumption i: taken too lightly, and 

I say that on the basis of common experience, and one that 
probably many of us share, the reason for a Head Start 
program is not because people have problems, or the kids have 
problems with tests, but had deficits in educational 
exp«rience, and we want to overcome that, anc because there 
might be an age difference betwee: one set of subjects and 
another set, doesn't mean that all the educational experience 
deficits are overcome. 


Q You mentioned in discussing your work in designing 


to 
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an examination for secretaries a concern with eliminating 
any testing for irrelevant skills, I believe you said, 
because of the impact this might have on excluding 
minorities? 

A That is right. 

Q What were those irrelevant skills? 

A Well, take the simple one. Many offices would 
like to have a secretary who types at the fastest rate. 

Now, there are som2 typince tasks such as tables 

and statistics where it is not the fastest rate you want. 
First of all, they are not produced that frequently, and, 
secondly, the primary variable is accuracy, and you would 
rather have someone who is terribly accurate in both reading 
«nd in typing than to have someone who was very fast as to 
typing, and that is an obvious one which always is or iad it 


is overlocked, just because speed seems to be most desirable. 


Another one might be a pleasant telephone voice, 


depending on where he or she, the secretary, is going to 


work. There are “acit ,inds of operat’rs and managers 
attitudes. 
Q Do you see any indication that that concern with 


possible irrelevant skilis might have had an impact here 
in excluding minorities? 


ry I don't see that, but I am not say..ng it is not 


.% 
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in existence. I didn't look for it. 
MR.MINTZ: I have nothing further. 
THE COURT: All right. 

REDIRECT EXAMINATION 

BY MS. OMANSKY : 

Q Dr. Williams, you were given the situation 
hypothetica’?ly in which people have passed in this position 
of supervising human resources specialist; they hac passed 
the previous examination, and they had been on the job 
for, let us say, a year or two years. 

Now, what would be the effect of knowing that 
people have passed a pricr examination on their evaluation by 
their superiors? Is that a problem? 

A If the superiors knew that it usually confounds the 
measurements. 

I responded -- maybe I should have been more explici 
-- 1 responded to the notion of the question that if someone 
gave tha exam at some prior time and you didn't know what 
the results were -- the supervisor didn't know what the 
results were, and so on, could you do a validation, and the 
answer is yes. 

Now, the term “pass” or “not pass", one doesn't 
hav e to put in a cutoff score; one could keep that tcit. 


Q But in this case, of course, there was some evidence 


Se 


14 MS. OMANSKY: And another thing, the situation 


15 where Mr. Mintz said the people passed the test and were on 
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2 that everybody knew that people had paszed the exam, is 
3 that correct? 
| 4 A That is a fuactional restriction. 
5 Q It is always a factor, isn't it, in rating somebody, 
6 that the knowledge whether or not they had passed a prior 
7 exam would color -- in addition to the ethnicity problem which 
8 you mentioned and the lack of a measurable criterion, this 
9 is also another problem, is it not? 
10 A Yes, it is. 
11 THE COURT: I take it you are bringing this out with 
12 regard to the predictive validation discussion, and my 
13 asking about that ? 


16 the job for a year, and why was it. 
17 THE COURT: That is what I mean. 
18 Q What effect would you give to an evaluation that 


19 says "Check whether this employee is satisfactory or unsatis- 


” factory"? 


21 How meaningful a valuation ‘s that? | 
22 | A It is usually too gross. 
23 THE COURT: I think Dr. Katzell has testified on that 
2A and I am aware of the obvious limitations. 
25 | Q Now, also on your cross ex7ination I think Mr. 
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categories for the knowledges and skills and really plugged 
in a description of tasks, now I am going to ask you, 


supposing there did exist a categorization, a prior 


categorization of knowledges and skills, and Mr. Rosenberg 
went out and did his job as he testified he did, 
and did a job analysis and came up with the same knowledges 
and skiils, would that coincidence, would that identity make 
his job analysis any less valid? 

A No. 

THE COURT: If you are concerned that my remarks 
indicated that I thought it would, I did not. I simply 
thought that if a man uses categories not on the basis of wha 
he has found *“t just on the basis of something lying around, 
that that cbviously wouldn't be sound. 

We all have to use concepts and words we know of 
in the past. 


19 
Q Is it possible that even on a test that has been 


20 
21 

, who scored very high in the test and that did not perform on 
22 


the job? 
A Yes. The validation procedure does not guarantee 
that everyone is going to score the same way. 


25 
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2 
Mintz was saying supposing Mr. Rosenberg had preexisting 
THE COURT: No matter what validation? 


preiictably validated, that you would still find individuals 
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THE WITNESS: Yes. That is just to be sure that 
the logic of inference is in fact. The phenomenom is not 
changed in your measuring. 

THE COURT: Doesn't it boil down to this, 
that you have a lot of imperfect instruments in your 
profession like we do in mine; the tests are imperfect; 
the validation is imperfect, and if you can without sacrific- 
ing other value and policies, validate in as many ways as 
possible, so much the better, if you can come closer to a 
truthful conclusion as to whether the test actually does 
do what it is supposed to do? 

THE WITNESS: That is correct. I just have one 
qualification. The purpose of systematizing ‘the procedure 
is to be able, one, to minimize errors in inference, and 
if they do creep into identify at what point do they enter 
so that you can take corrective action. 

MS. OMANSKY: No further questions. 

THE COURT: Thank you very much, Dr. Williams. 

(Witness excused.) 

THE COURT: We will take our morning recess at this 
time. 


(Recess.) 
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MS, OMANSKY: I have no further witnesses. 
MR, MINTZ: In rebuttal, the plaintiffs 
call Dr. Felix Lopez. 
FELIX Me LOPEZ, called as a witness 
in rebuttal by plaintiffs, being first duly sworn, 
testified as follows: 
DIRECT EXAMINATION 
BY MR. MINTZ: 
Q Dr. Lopez, woul¢ yon give us your educational 


background first, pleas? 


THE WITNESS: No. NYU. 


A I have 4 masi:c'sa degree in public administration 
and Ph.D. in personne. psychology. 
Q From what school «ad when did you obtain 
that? 
A From Coiumoia ‘jniversaty. 
THE COURT: 3oth degrees? | 
| 


THE COURT: Which is which? 
“THE WITNESS: NYU for the master's in 
public administration, andthe Ph.D. from Columbia. 
I am also a certified psychologist in New York 
State, and I am diplomate in industrial and organizational 
psychology of the American Poard of Professional Psycholo~ 


gists. 
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2: QO Could you give us your occupational experience? 
3: A Currently, I am piesident of my own consulting 


firm specializing in personnel management, particularly in 


restructuring employment systems of many governmental and 
private employers, 


Before that I spent 20 years with the Port of 


New York Authority. I was manager of selection and 
8 placement for 16 years, and then was manage~ of manpower 
10 planning and research for five years, aii “ben I retired. 
u I am also an adjunct professor of organizational 
12 behavior in the Graduate School of Buriness Administration 
at Long Island University. 
14 Q Would you state your professional associations, 
15 please? 
16 | A Well, I will state some of them. Iama 
W member of the American Psychological Association, the 
18 New York State Psychological Association, the American 
19 | Management Association, the International Personnel 
0 | Management Association, the American Society for Training 
| 21 | and Development and so on. 
2 Q Are you, incidentally, a member of Division 14, 
23 | Division of Industrial Psychology, of AAPA? 
} 2 | A Yes, I am, 
2 


Q What have you published in the versonnel selec- 
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tion field? 

A I have written four books. The first book 
was Personnel Interviewing, Theory and Practice, which 

was inl : It is in its second edition now in the 

process of writing. It will be published later this 

year. | 

I also wrote a book Evaluating Employee Per- 
formance, published hy the Public Personnel Association. 

t d#d a monograph called Evaluating Executive 
Lecision-!'aking, published by the American “Management 
Association, which was a study of the decision-making 
abilities of about a thousand managers; and I did a book 
called The Making Of A Manager, in which I put down my 
theories of selection, managerial selection, supervisory 
managerial selection, 

I have also published in many journals, the 
Public Personnel keview, Personnel Psychology and many 
others, 

Q We are dealing primarily with two main occupa- 


tional positions in the Port Authority and your present 


personnel selection, or have you donein those positions? 
A In the Port Authority I was responsible for 


the selection of all employees. We had our own merit 


firm. What sort of things do you do relating to 


system, not being under the New York Sting law, because 
we were a bistate agency. We had our own merit system 
that I developed and I had a staff that designed, con- 
structed examinations, analyzed jobs, established 


eligible lists and promoted people from these lists. 


But being that we were not under the state 
requirements we were able to introduce a great many 
innovations into the practice which then occasioned 
my association with many other public agencies throughout 
the United States at the state, federal and local 
level, principally, I would say, and most extensively with 
the City of New York Personnel Dep ~tment, where since 
1953 I have almost continuously been involved in the 


construction of and administration of examinations for 
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| principally personnel positions, I would Bay. 
to construct examinations for the personnel depart .ent, 
they had to bring in outside experts, and so I served in 
almost every capacity from item writer to serving on the 
Committee on Manifest Errors, 

I also testified in court for the city in the 


state and Supreme Court. 


Q In addition to your work with the City of New 
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York, have you had association with other public agencies 
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in personnel work? 
A Yes, in the State of New York, the State of 
New Jersey very extensively -- 
THE COURT: What have you done for the 
State of New York? 
THE WITNCSS: I constructed tests for then, 
I sat on oral boards for them. I have also conducted 
training programs. 
THE COURT: Am I to understand generally this 
has been in the personnel field? 
THE WITNESS: Exclusively in the personnel 
field. Not the tests, though, in the personnel field. 
THE COURT: I meant the tests. In New York 
City the tests were for people in personnel, is that 
right? 
THE WITNESS: I also did tests in the city 


for other agencies. 

For example, I served on the selection com- 
mittee for personnel director of the Human Resources Agency 
and I have done other general administrative positions. 

I have also helped with the firemen's examination. 
Q Have you had any occasion to work with or 
be associated with the United States Equal Opportunity 


Comnissi »% 


_ 
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2 A Yes. I was one of the first psychologists, 
3 I think, to study the differences in test performance 
4 betwee... whites and blacks, which was very cbvious to us, 
5 and to explore the reasons why that would be so and 
6 what effect that had on predicting job performance. 
7 I did a study of the toll collectors in the 
8 Port Authority and found tiiat while the differences were 
9 there they had no effect on four criteria of performance, 
10 and that study was published and became somewhat con- 
ll troversial, but the data was still there. 
12 The Equal [Cmployment Commission or its predeces- 
13 sor was very interested in what I was doing, so I was 
14 invited to serve on a res@arcl: cormittee under Hobart 

| 15 Yaylor, who was appointed by President Johnson at the 
16 time, and gave himsome advice on ‘title 7 before it was 
17 enacted into law, and since that time I have worked on 
18 and off with the Equal Opportunity Commission in various 
19 capacities. 

| 20 Most recently I have been advising them and 
21 giving them my thoughts on the new uniform guidelines that 
2 were referred to yesterday. Immediately before that 
23 I served on atask force in evaluating the selection 
2A procedures of the telephone company. I was a govern- 
25 


ment witness in that FCC hearing, 
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"Re. MINTZ? That covers the initial questions. 
I offer Pr. Lopez as an expert in the field of personnel 
selection. 

“"S. OMANSKY: No objectis 
BY MR. MINTZ: 

fa) Do cecall when I originally contacted you 
regarding this © stter? 

A I believe it was in November or October, some= 
thing like that. 

9) Nid you subsequently agree to examine materials 
relating to this case? 

A Yes, I did. 

9) Did you have any conclusions or any opinions 
about this matter before you examined those materials? 

THE COURT: What matter? 
MR. MINTZ: The matter at issue here, 
the lawsuit and the issues involved in it. 

A Well, I expected to find what I did find, 
because of my familiarity with the testing procedures of 
the city and having discussed the issue with various 
officials from time to time. 

THE COURT: You mean before you examined 


the tests, after you had heard from Mr. Mintz what the 


results were in terms of percentage of groups passing 
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and failing, and so on? 


THE WITUESS: he didn't give me any details, 
He said -- 

THE COURT: What did vou know? 

THE WITNESS: lie asked me would I testify 


as a witness, and I said yes, I would, and <-- 

THE COURT: If this question and answer is 
to mean anything to me I have to know what you did know 
when you expected to find what you fonnd, 

THE WITNESS: I think that will come out in 
mv testimony. What I found was not a surprise to 
me, I felt, so that if that is the point of it.-- 

THLE COURT: It isn't sufficient, and I am 
sorry, for me to understand what you are trying to bring 
out 

MR, MINTZ: Let me try to clarify that. 

THE COURT: I take it when Mr. ‘tintz met 
with you he told you something. What did he tell you? 

THE WITNESS: He called me on the telephone 
and he asked me, he said he had a case involving a city 
examination in which there was a question about the 
adverse impact on a particular racial group. 

THE COURT? I take it he told youwhat the 


job category was, or something like that, or did he not? 
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THE WITNESS: lle did, but I didn't remember 


or recall it. 


THE COURT: You knew they were not motor- 


truck drivers? 
THE WITNESS: I knew it was “he Human Re- 
sources. 
SOURT 8 That's what I am asking. : 
THE WITNESS: Right. 
THe COURT: Once you heard that he had a 
case in the pattern that we Have beoome accustomed to 
now, once you lieard about that, you then, when you saw 
the test, were not surprised at what you saw, is that it? 


THE WITNESS: That's right. 


Q You have had extensive prior contact with 
testing procedures of the City of New York, Department 
of Personnel? , 

A Yes. And with many other public personnel 
agencies, I think I am pratty familiar with the whole 
thing of public personnel selection and their use of the 
competitive written examination as a major tool of 


selection. 


Q It is that prior knowledge that led to 


these preconceptions that are borneout, is that correct? 


A Yes. 


_ 
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Q What material did you in fact examine in -repara- 


tion for your testimony here? 


A I don’t know what exhibits you call them but -- 
Q Identify them by name. 
A I reviewed a field audit report dated February 


4, 1972; I reviewed the examination No, 2013 and the key, an 


a 
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examination No. 1631, th a. »uncement of the jobs with 
the City of New York for supervising human resources 
specialist, and I reviewed ard read the depositions of 


r. Rosenberg and Miss -- what's her name? 


Q Willingham? 
A Yes. 
Q You were also present in court during the entire 


day ye ttc1day for Dr. Katzell and br. Williams’ testimony? 


A Right. 

Q On che basis of your examination of these 
materials, would you first state what conclusions you 
came to and tnen I will ask you to explain how you came 
to those conclusions? 

A Okay. I came to three conclusions, My 
first conclusion was that the alleged job analaysis 
does not even 1« . ely meet professional standards, 

I will explain whyin a minute. 


Secondly, I came to the conclusion that the 
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written test was inadecuate as a measure of any portrait 
associated with performance in the job. 
Third, I came to the conclusion that there 

was no evidence for content validity or any other validity 
of this test. 

Q Let's focus on what's ween described by some 
here as a job analysis, the field aucit report officially 
denominated Defendants’ Exhibit F. You say you found 


that inadequate? 


A Yes. 

fa) What is the basis for that conclusion? 

r. First of all, let me clarify some ideas 
here, There has been a lot of words used in a rather 


imprecise fashion that I Lave heard heard yesterday and 
today. When we talk about job analysis, that is a very 
broad term and it refers to four distinct procedures. 
One is the minute, microscopic task analycis, motion 
analysis,’ imetables, done by industrial engineers. 

The second type is a work-flow znalysis that 
is usually conducted by systems analysts. 

The third is a rather broad, generalized 
analvsis conducted hy compensation and classification 


people for the purposes of slotting a job in a specific 


salary range. 
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the last one is what we call work oricnted jov 

analysis, which is performed to determine the human 

qualifications or the traits required to perform a 

given job. 

In selection we ae not measuring the job or 
an area of a job as I have heard over and over again in 
the past couple of jobs, we are measuring people and what 
traits they have to behave in a certain wavy which we 
call job performance, 

THE COURT: What do you mean by saying in 
selectionyou du that? 

THE WITNESS: In selection, We are not 
rm asuring, we are not trying to measure a job area or a 
job function, you see, We are measuring a 
person to see whether he has certain qualities, we 
call traits, to perform in a certain fashion. 

So the job analysis that has to be conducted 
is one that identifies the traits that a person has to 
have. It is generally agreed in the literature that 
types A, B and C, the first three types of job analysis, 
are inadequate for selection purposes, for the design 
of instruments to measure human traits. 


What we need is worker-oriented job analysis, - 


and there are some systems now. One of them was men= 
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tioned here by Dr. Williams, the critical-incident 
method, There is work going on for years in Purdue 
University by a psychologist by the name of McCormick, 

I have my own system I use that is a modifica- 
tion of a system used in the federal service by a man named 
Primoff, a psychologist named Primoff. 

So this is what is necessary, and it became 
crucial in this issue of adverse impact. You have got 
to begin with worker-oriented job analysis. If you have 
a fuzzy or an inadequate idea of what is required to 
do the job there is no way that you can develop an 
instrument that will measure this. 

I also heard that this is an imperfect science. 
Well, it isn't that imperfect as would be indicated by 
what I see presented by the city. It is a question 
of selection strategy, that is what we are after, and 
this is the issue, as I see it, in this particular case, 

Q In what respects does the field audit report, 
Defendants’ Exhibit F, fall short of what you would con- ik 
sider adequate for the worker-oriented job analysis you 
say is required here? 

THE COURT: I will need a copy of this to 
follow you. 


THE WITNESS: I think you might. 
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(Pause,.) 
A First of all, in my opinion, the sampling -- 
and we have heard this was inadequate to sample only, 
I think it was, four or five positions, and we must 
distinguish between a position and a job. Every 


position differs frem every other position in some slight 


detail, 4 job is a class of similar positions that we . 
can bring together for purposes of pay, training and 
selection, 

Therefore, we have got to sample a number 


of positions, When I go in to do a study I sample a 


broad spectrum of positions, not just four or five. 
THE COURT: The number that you examine I 
assume has some relationship to the number of people 


that will be employed. 


THE WITNESS: Yes, that's right. 
THE COURT: Or are employed, 
THE WITNESS: Well, the number of positions 


you have and the varying nature of them, 

THE COURT: But you mean,taking into con- 
sideration the number here, four was insufficient. 

THE WITNESS: Yes, it would be inadequate, 
I think, because -- and it was only limited because of 


the city's policy, You see, the city has a number of 
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unioue policies that inhibit good selection, This is 
one of the fundamental problems here. This is one of 
them, not talking to somebody occupying the job because 
you may in some way indicate the content of the examina- 
tion. 

THE COURT: What did you do about that in 


the Port Authority? 


THE WITNESS: lirst of all, we would never 
let this situation arise. We would never permit anyone 
to occupy a position provisionally for a year. Three 


months was the absolute limit and we had to come up with 


an examination and select Lefore thats just so you would 
not get this kind of problem later. 

And this is not the first time this has 
happened in the citv. This happened in 1953, when I 
cane in, I think there were 10,000 ,,rovisionals or more 
in the city when Mr. Sehocht-r became the first personnel 
director. But you have the problem now,but you must -=- 
you have these positions, you have to talk to the incum- 
bents to get a representative sample. 

So this was the first inadequacy. 

.@) You will recall Mr. Rosenberg also stated it 

is indicated on the face of the document and in his deposi- 


tion and in his testimony that he interviewed supervi- 
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sors -~- that is, superiors to supervising hunan resources 
specialists -- to ask chem about the nature of the job. 
Would that in your opinion be an adequate substitute for 


sneaking to the incumbents on tiie job? 


A Not necessarily. Provided they had once 
perforned that job. But we do find gans between what 
a supervisor says the job consists of and what the in- 
cumbent says, so we always include both, and sometimes 
personnel people, though we get resistance to that be=- 
cause the supervisors all agree that they don't know 
what the job consists of, 

hut we like to get incwabents because they 
have a view, a perception, and we like to get super- 
visors. 

We also like to observe the job in question, 
but this gets you intothe problem of sampling again 
wecause you can observe soembody on a single day and you 
might not see anything that is critical to that job. 

The second inadequacy, and a more serious one, 
is the lack of any methodology to assure reliability or 
establish any oLjectivity or any comparability in this 
analysis. This is purely subjective and it is 
ratier -- it has a very crude notion in it I will point 


out in just a moment, I don't really understand what 


®R F§ BS B 


inda Lopez@~tirect 576 
it is trying to -- what it purports to say or how you 
can relate it to the traits neccessary to perform this 
Jon. 

But there are, as I pointed out, a number of 
methodologies that have been developed. Some are bet=- 
ter than others, All of them have some imperfections 
but you do get objectivity. A little later perhaps 
I can explain it ina little more detail, 

The third one, and the most serious one, just 
by looking at the content of this, which would make anv=- “ 
»ody concerned with testing or neasurements throw up his 
hands, you get knowledge and skills required, if you 
look at the bottom of page 2, and you see all sorts of 
imprecise and ambiguous terms being used. 


For example, the one I have here says. "Knowl- 


edge of," What is knowledge of? What level of knowl- (|< 
edge? It just says "Knowledge of functions." What 
functions? That doesn't tell you anything. You can 


write anything there and say, well, that's measuring 
knowledge of functior3. 

So if you say knowledge you have to define 
what you mean by knowledge and what the level of it ° 


is. We generally have a standard, We say knowledge 


that would ordinarily Le acauired by two years of normal 
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training. 

Then they say familiarity with. We would 
say what's the differenc:: between familiarity with and 
knowledge of. 

They alternate these terms, What is 
familiarity with? Is it simple recognition or 
something like that? 

Then we get to words like skills. What 
kind of skills? Shills in deciding problems or 
a’ility to supervise. 

Now,we will commenton that one a little later. 
What's that notion? That is the construct concept. 

So you see tie loose use of these words just 
doesn't give you any guide, definitive guide, to develop 
a measure, To say that these we will measure now 
in a written test is making a verv giant leap forward. 

If that were handed to me I would say that 
this is useless in developing a test plan. 


e) As you know from what you have read, Vir. 


Rosenberg had many yea. 2 of experience in doing desk audits 


for the purpose of classification, His testimony 
was that this was the first audit he did for the purpose 


of developing a selection device to transfer from one 


division to another. Do you feel in view of the differ- 


ay 


10 


11 


12 


13 


14 


15 


18 


19 


25 


¢3 


rida Lopez-direct 57 


’ 


ences in those purposes there might have been a pro}-lem 
inherent in that situation? 

Yes, that is tlhe heart of the problen,. 
That tiie city has heen trving to do is adant a method 
that was suitable for compensation purposes, where the 
precision is not really essential, you are just doing 
some gross classifications of people, and even here 
sone of this is questionable but, you see, to adapt this 
to the much more precise task or purpose of selection 
results in a monstrosity. 


Q Sor a selection-oriented job analysis do you 


need more information, less, or is it just a different 
kind? How does it work? 
m You need a lot more information than is pro- 

vided here. You need more than just a ficld audit, 
onserving a jol. 

For example, if I can give you an example ef 
of wiiat I think is a proper way to do it, in con- 
structing an examination for battalion fire chief in anothe 
city we went down there and we are the test constructors 
now, not a third party, but the test constructors 
vant down and ve first interviewed a great many in- 


cumbents and supervisors right up to the fire commissioner, 


we spent a day in the firehouse, ve went to a fire and we 
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saw how they handled directly -- very fortunately there 
vas a big fire in the city that day so we were able to 
get a good example of it. 

And then we had them complete a questionnaire 
call.id a position analysis questionnaire, which is 
about 18 pages or 20 pages, and which goes into not only 
tre task but the demands of the job. That is one of 
the big things that is missinc from this and which is 
a cricical factor. those are the things you have to 
put up with, the job conditions; working in the cold, 
working under hazardous corJitions, 

You analyze all of that, We took t ec ques- 
tionnaires and sent them to Purdue where they went through 
a computer and came out with a profile of what the jok 
trait: required to perform that job. With that we now 
nad == and that was compared to what they had in their 
computer, 10, 15, 20 thousand jobs, They do it very 
reasonably because you male a contribution to their data 
pool, With that now we were able to Say . 8S, one 
of the factors that has to be measured is the ability to 
make decisions. 

We then said this is the measure we will use 


to evaluate that. 


THE COURT: What do you mean by the ability 


¢ 


ur 


6 


mda Lopez-direct 580 
to make decisions? 

THE WITNESS: The ability to decide, to choose 
a course of action among several options, 

THE COURT: The reason that I ask is because 
while I think I understand what that phrase means, it 
seems to me it also hasthe same problems that No. 5 here 


does, ability to supervise and direct. 


THE WITNESS s Yes, you are right, but we de- 
Fined it, you see, very precisely. There is about 


three paragranplis. 

THE COURT: You are just summarizing? 

THE WITNESS: Yes. We would say decision- 
making ability and then list it, 

THE COURT: And indicate what that meant? 

THE WITNESS: We listed it very carefully 
and then the level of the ability, how complex, because 
a stenographer or clerk makes decisions but they 
are not of the same order of magnitude or complexity of 
an assistant fire chief. 

So we had to spell that out and develop a mea- 
sure of that. 

But unless you define it precisely you can't 
develop an instrument. 


BY ' Re. JSILTTZ3 


9) Who were the people who filled out this 18- 
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page questionnaire? Were they incumbents, supervisors 
or both? 

A Both. And all did it independently without 
consulting one another. 

Q Do you recall in that instance how many in- 
cumbents in the job you actually interviewed or submitted 


the questionnaire to, or both? 


A Well, I estimated all told maybe 40 or 50’ 
people. ) 
THE COURT: How large a pool? 
THE WITNESS: I am guessing, but I think 


in this city they had about 100 battalion chiefs, 

THE COURT: So that would be a prett: sub- 
stantial percentage, a pretty substantial sample? 

THE WITNESS: Yes. 

THE COURT: Have you got an opinion as to 
how many incumbents you think should have been reviewed 


in this situation here? 


THE WITNESS: We have a strategy for determin- 
ing that. We just don't guess, 

THE COURT: Would you tell me what your strateg 
is then? 

THE WITNESS: You develop what we call a 


job matrix, you see, and we determine <= and that is 
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determined bv location. and by tyne of work, and then 
how many are in each of these, and then we would tal.e a 
10 per cent sample or something of this sort. So it 


ould depend. 


Se a One of the important things we do with this 
eadors 


is that we can then meusure the interrater reliability. 
If we get a very low reliability -- in other words, if 
they aurte, this is the function of tle independent 
judgments and they agree. If you don't get agreement 
we call them back and say something is happening 
hecause we find frequently that jobs with the same 
title are quite different, but that for compensation pur- 
poses they give it a label, hut they require much 
different traits. 

Sometimes we find jobs with different titles 
are the sane. $0 this is the process of discovery that 
is so crucial to develop an adequate selection strategy, 
and that is what I think is required by the guidelines, 
At least that is what is intended. 

8) Just for clarification, when you say “we" 
do you mean that a team of people perform the job 
analysis or was it yourself? 

A No, it is a team, You also have time, 


and we want more than one person on our side involved 
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2 in it. Just a single perrorn would introduce a 

3 great deal ¢-" the subjectivity into it. 

4 QO If only one person is doing the job analysis 
there might le an introduction of J.i-ses whic! vouldn't 
6 be corrected ly others? 


7 A Ye&. 
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Q You mentioned you don't think it conforms with 
the guidelines. What were you referring to? 


A The proposed guidelines which require an adequate 


job analysis where there is an adverse impact. 


Q Proposed federal uniform guidelines? 
A Yes. 
This was the critical issue in the AT&T case. They 


had no job analysis. 
| Q Those guidelines are the ones presently being worked 
out between E' LU, U.S. Civil Service Commission, Office of 
Contract Compliance and Department of Labor? 
A Yes. 
And they have been reviewed by the municipal 
and state agencies, public agencies, too, who have objections 
to them. 
Q You are familiar with the presently existing 
EEOC guidelines, also, I assume? 
A Yes. 
Q Is your conclusion the meaning of job aazalysis as it 
appears in that analysis the same? . 
A It has been clarified because of the fact ina 
number of cases rather inadequate studies of the job have 
been submitted as meeting the requirements oi the 


present. guidelines is to @pell that out in more detail 
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as to what is an acceptable job analysis. 

Q How many occasions have you had to construct an 
examination for selection to a particular position? 

A Innumerable. I would say over a thousand. 

Q Do you feel that you could construct an adequate 
examination on the basis of this job analysis? And you have 
also seen, I believe, the test plan, the document which gives 
the traits and the percentage weights? 

A Yes. 

Q Do you feel that you could construct an examination 
from that information? 

A No, I couldn't. 

Q Why not? 

A Because it doesn'tgive me any indication of several 
important pieccs of information, The first is that it 
doesn't tell me what are the actual traits, what is it that a 
person has to have within him, you see, to perform the job 
effectively, which are critical. In other words, without 
that ability or adaptability I couldn't perform the job at al 
effectively. 

Third, I need to know the weight, the relative 
weight of the different: traits that are required. This is 
another area in ‘inbeonian that you oferemphasize one trait 


and to the exclusion of another which may be the one that 
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makes the greatest contribution to the job. 

For example, take a telephone operator. If you 
can give her a mental ability test, as was done by the 
Telephone Company, and it did predict success in the 

training program, but that also predicted turnover. In other 
words, the brightest person, the people who got the high- 
est scores on that test, they did well in the training 
program but they had no adaptability to the working con- 
ditions on the job and they quit, and so the company had a 
very high turnover. 

Had they emphasized adaptability to repetitious, 
fairly monotonous work, they would have had stabilized 
the job and still had people who could have performed it 
effectively. 

Q The field audit report purports to give weight 
to the various factors by the portion appearing at the top 
of page 2 which lists things that are done on the job 
and then by number indicates which skills are relevant to 
those things. This is further translated into percentages 
on the test plan. 

Why does thatnot, or is that not adequate? 

A It isn't adequate because, first of all, I can't 
figure it out to start with. That doesn't mean it is 


inadequate , it may mean I am, but certainly ought to be 


22 
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Clearer than that. iie says through the interviewing learned 
about negotiations for funds to permit on the job develop- 
ment of disadvantaged residents of poverty areas of 
New York City, and then he has numbers after it, one, two, 
three, four, five and eight. Presumably these are the 
knowledges and skills required to negotiate these funds. 

I am assuming then that in some way he collected 
the number of times No. 1 was required and assigned that a 
percentage in his test plan, and that is a pretty rough 


approximation cf an analysis. 


As I said, I am not sure of that because I don't 
think the percentages add up then, so I really don't 
know how they came out. I have seen nothing to show me 
how they came to the conclusion of those percentages or 
arrived at those percentages. 


Q Assuming that in fact it was a simple matter of 


counting the number of times each of those traits was necessary 
to perform each of the tasks found in the job, would that 
be adequate? 

A You can do a frequency analysis, the frequency 
with which something is done, but it might be something 
that is done frequently that you don't even have to test for 
because everybody can do it. For example, you may find 


that a person, the thing they do most frequently is answer 
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the telephone, but almost anybody can answer the tele- 
phone, so that is not a critical factor. 

It might be what they say to certain people who call 
which becomes a critical factor, so frequency by itself is 
not sufficient to develop a test plan. 

Q How about the fact -- take the example you used, 
negotiate for funds. There is no indication of how often 
a supervising human resources specialist negotiates for 
funds. Does that create a problem? 

A Yes, that does. And also what do we mean by 
negotiate for funds? What specifically are you doing here? 
That is what we are after. Because a lot of people can 
say they negotiate becav e they participate in the negoti- 
ation, or they participate in the discussion. There are 
different degrees and levels of negotiation. 

Q If you in conductirg a job analysis were to find 
that the people you are trying to select in fact do have 
to engage in negotiation for funds, how would you go about 
identifying the traits that they need to successfully 
perform that test? 

A Well, I don't want to go into a long, involved, 
technical discussion here, but you have to have some 
preconceived plan, some established way of taking 4 teat 


and breaking it down into certain traits that are necessary 
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for job performance, and then those traits or constructs, 
if you will, would be whatyou would measure and you would 
develop the measures for that. 

So it is taking a job fuiction -~ and this is a 
function here, a task to negotiate funds -- and then 
determining, well what does it require in the way of hu- 
man traits? 

We can say, well, first, it s require speaking 
ability, but it might require a teterance, it might require 
influence, and many other traits that you could name. 

Each system has its own approach to this, its own termin- 
abgy. 

THE COURT: To some extent isn't that what Mr. 
Rosenberg was doing? 

THE WITNESS: Yes, in a very rough way he was 
attempting to do this. 

THE COURT: The ones, twos and threes, meaning 
this quality or whatever you want to call it is necessary 
in regard to this function, in his opinion. 

THE WITNESS: All I am saying is that it is so 
crude -- 

THE COURT: I must say I do understand the differ- 
ence between the depth of your approach and the way he 


testified that he did it. I just wanted to understand if 
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it is qualitatively the same approach. 

Q Was there anything else that you felt was 
relevant to the job analysis to suggest its inadequacy, the 
field audit report? 

A No, I think I have covered all the points I want to 
make. 

Q You also stated you examined the tests themselves 
at issue here, 1631 and 2013, which are identical except 
for the last ten questions. What were your conclusions 
about the test and what was the basis of those conclusions? 

A Well, overall I felt it was a bad test. Ef it 
had been submitted to me for approval I wouldn't have 
approved it. Just from the standpoint of construction. 

It has many, many ~-- and it was pointed out yesterday 


there are many inadequacies, and this is a question of 


relativity. 

Pirst of all, it is a multiple choice test, and a 
multiple choice test in itself is subject to severe 
limitations that have been written up in the literature. If 
one is to write a multiple choice item you have got to 


have a great deal of expertise and you have got to do a good 


deal of research if you want it to be effective and working 


for you. 


One of the principal problems with a multiple 
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choice test is that it tends to pena ze the expert 

as much as the incompetent. When you read these, as I 
did, without looking at the key, trying to select the 
right answer, I in 24 out of 25 questions picked the 
answer that I knew the examiner was lookingfor, although 
I could see in almost every question another another 


that was either better or as good as. 


Q Do I understand you administered this test to 
yourself? 

A Just the 25, The reading comprehension and the 
graph. 


THE COURT: How did you do? 

THE WITNESS: As I said, I had almost a perfect 
score. 

THE COURT: On the basis of your being able to 
guess what the examiner was looking for? 

THE WITNESS: I know what the test constructor 
is doing, and why certain items are in here, and so forth. 


Now, you raised the question, you see, and the 


question has been raised tere, well, it ia impossible to tell 


in advance how effective a question is. That is not so. 
The City of New York is unique in several ways 
because there is a peculiar set, mental set here in the 


city both on the part of those who take the test and those 
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who administer them, and all they are concerned about is what 
the judge will say, because they assume that almost every 
examination is going to go to court. 

So when you talk to a test constructor -- 

THE COURT: Why don't they show it to me ahead 
of time? It would save a lot of trouble. 

THE WITNESS: That is right, it would. So they 
are concerned not about the validity of an item but 
whether it will stand up in court when in the cross examinati 
someL:dy takes and looks at an itema@md says I think this will 
be the right answer. And I spent many hours, two days on the 
stand, defending seven questions that I didn't write. 

One of the problems therefore is they cannot verify 
their items in advance, but others can do it. For example, 
we develop what we call an item file. 

Q Incidentally, before you go into that, was there any 
particular basis you had for that feeling that the city's 
exams are largely based on concern for protecting them from 
attack? Did anyone say that to you? 


A In conducting an oral for chief examiner a few years 


and he said “No appeals.” 
Q Who was that that said vhat to you? 


A I won't mention his name. 


ago I asked what is the characteristics of a good test, 
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THE COURT: The Corporation Counsel. 
THE WITNESS: I didn't hear that. 

A And so, also because of this fact I have been 
caught up in it myself and I don't blema ethem in a sense 
for being concerned about this, but does he destroy the valid+ 
ity of the test, and it inhibits their ability to innovate, tq 
come up with new ideas in testing. They have done some. 

THE COURT: I understand why you believe thz*. a 
concern about the conformance with the legal standards as 
the Court's see it, let us put it that way, on the part 
of the city may have a distorting effect, if that is reaily 
not what you are testing for, but I don't understand what 
you mean when you say it is not possible for the test to be | 
tested in advance. 

THE WITNESS: That is what I am going to show, | 
that it is possible. But they don't do it as a matter of 
policy. 

THE COURT: That is just a matter of policy? 


THE WITNESS: Right. 


THE COURT: You weren't connecting that with -he 
fact that they were trying to avoid unsatisfactory litigation 

THE WITNESS: Yes, they are, Because, you see, 
they have to -- they suspended this policy for a period of 


time, but once the test is administered they have to give the 
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test to the people who took it to study it, and that means 
of course that you cannot reuse those items, you see, and 
they are so cautious about this they wouldn't <o any pre- 
testing. They wouldn't pretest it. They are afraid 
somebody who failed the test would allege that one of the 
persons leaked the information of what was on the test. 

THE COURT: I see. 

THE WITNESS: They say it will violate the security 
of the test, and so they go to great lengths to maintain that 
security. 


THE COURT: Of course that is an important con- 


sideration. 

THE WITNESS: But it is not an insurmountable 
one. 

THE COURT: Good. Let us hear how. 


THE WITNESS: Well, first of all, what a great many 
public agencies do is they maintain item files that measure 
various traits, areas of knoweldge. We had them. We had 
a file of some 15 to 20,000 questions that we collected from 
all over so that the test constructor after he took his 
test plan could then go and pick out items. 

He would go to a file, and on that card it has this 
kind of information about the item -- when it was used last. 


And we had a policy that you couldn't use an item if it had 
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been used within the past two years or something of this 
sort. 

Secondly, for whatjobs it was used. 

Third, we had some of the information that we 
saw yesterday, the item tally. 


Now, tnat is where we get the discriminating power 


of the item, which is the coefficient of correlation. You get 


the 25 per cent and how they did, and the bottom 25 per cent 
and accrue a set of tables. I could have done that for this 
document here, this item tally, I think using Flannigan's 
tables in a half hour. 

We had standards for that. We would say that if an 
item did not have at least the correlation of point 30 we 
restricted it becaugeit was not discriminating. Then the 
second thing we had is the item difficulty, the percentage 


of people who selected the right answer. 


Now,you asked that question yesterday about what 
standards. We had standards. If an item was answered 
correctly by over 90 per cent we rejected it. It was what 
we would call the who will miss it question. 

If an item had only 10 per cent people getting it 
correct we would reject it. We would only permit the 
test constructors to use a small number of items ranging in 


difficulty between 80 and 90 and 10 and 20. 
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We tried to get a difficult level of somewhere 
in the 60s for the test. So this is the way you prepare a 
test. 

THE COURT: I think Dr. Katzell testified about 
the desirability of having some questions purposely more 
difficult than others so that you could produce this spread 
that you want for civil service lists. Do you care to 
relate your theory abeut not having questions too difficult 
or too simple. 

THE WITNESS: You could have test items of the same 


level of difficulty and you will get a spread because you 


are presuming in your applicant population there is a 
spread. distributed. It may not be distributed normally, 
it probably shouldn't be, but nevertheless I don't think 
that is a concern. 

By giving casy items in the early part of the test 
we were sort of giving the test taker a chance to allay his 
anxiety and natural tension. We also had some difficult 
items in there to give a premium to those who were better, 
who knew more than the others, so we varied them. But we 
tried to get an average level of difficulty for that test, 
ard thet is one of the measures that when a technician 


brought an examination in for approval I would look at that. 


THE COURT: Excuse me, I interrupted you when you 
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were talk. j about the question of difficulty, 

THE WITNESS: As I say, this is some of the in- 
formation that would be on that card. So even if you were 
not able to do some of the predictive or concurrent validity 


| 
| 
studies that are desirable in public service selection you can 
use, you can construct the test made up of items that are 


verv effective and that have been used before and you know | 
wha. they are doing, you fin. t whether there is an option | 
that is misleading or confused or is equally correct, and ‘ial 
makes test construct %n much simpler. 

As a matter of fact, I had a contract with ETS 
to put this all in a computer at one time. Therefore, 
it could be done. 

Then after the test is over you can easily validate 


it, you can collect data on it. I have done this many, many | 


times. You can get predictive validity and you can also 


| 
| 
| 


get item validty in this way, so that you knew what was ) 
going into that test, you knew what it was supposed to measure! 
and you knew with what effect. | 
THE COURT: How would you proceed to validate it af-| 
ter the test was over? 
THE WITNESS: There are two ways of doing this. 


One way is to obtain performance evaluations at the time | 


persons are tzking the examination. In our procedure 
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and in other jurisdictions that is done automz ically 
because perlormance evaluation is part of the examination. 
THE COURT: Are you concerned about the problems 
that Dr. Williams raised in that regard this morning? 
THE WITNESS: They are not serious pro}hlems. 
They are problems. He points th2m out. They are not such 
as to deter. There are ways of overcoming them. 
I feel I have a way I was using the University of Chicago 
they use a different method, put ve do it all the time. 
That is one way of douanmrit at the time of the examination, 
because that gets you away from the contamination 
problem he refers to, that the greatest judgment would be 
contaminated by knowledge of how you did in the examination. 
The second way is the follow up six months later, 


and I have done that, and I haven't had any serious problems 


This could be done. You see, these things are 
possible and are being done in other public jurisdictions, 
and in my opinion with the current legislation and so forth 
they will have to be done. 

Q Are there other public agencies that you. are fanilia 
with which in fact don't release their tests and are able 


to review items and are able to review their items in 


of contamination at this point. 


that way? 


} 
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A Are their public agencies? 
Q Yes. 
A Yes, the State of Michigan does it. I know 


California. And of course there is a service you can 
subscribe to, the Public Personnel Association has a test 
service for a lot of small jurisdictions, and they will 
send you items. I never use their items because they 
appear on so many tests all over the country that they 
get abused, but they are used. I didn't originate the 
item ifle concept by any means. 

THE COURT: Dr. Lopez, would you mind if we went 
to lunch at this time. 

THE WITNESS: I would be delighted. ~ 


(Luncheon recess.) 
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AFTERNOON SESSION 


2:00 p.m. 


THE COURT: I just thought I would tell counsel 
I brought a new book of paper, but I hope I won't have 
to use it all. 
PE Eos M. LOPE Z, resumed. 
DIRECT EXAMINATION CONTINUED 
BY MR. MINTZ: 

Q Dr. Lopez, when we recessed I recall we were 
discussing the examination itself which you indicated you 
found to be -- I wont say invalid, that is a technical 
term - but you didn't find it to be a good test? 

A I thought it was poorly constructed. 

Q Poorly constructed? 


A Yes. 


Q What additional factors did you find about the 
test itself? You said, among other things, you administere 
a portion to yourself, and you got the right answers by your 
own estimate of what the examiner was looking for. Do you 
feel that someone with less knowledge of the workings 
of test examiners in general and examiners in the City of 


New York, in particular, might have more problems with those 


qestions? 
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A Yes. I think that the examinee is faced with the 

difficulty of trying to understand the point of the 

question, what the test constructor is trying to get at. 

Then, of course, you know, in constructing a 
multiple choice test, you have got to have, of course, 
the correct answer in there; and then you have to have 
other options or alternatives that are plausible but incor- 
rect. 

There is no sense in putting in three, you know, 
totally false answers because almost anybody can perceive 
them. 

So this puts the onus on the test constructor to 
make those other options that are wrong plausible. 

If he uses the system which the City uses of 
giving, of having only one right answer in the four and 
no other -- there are other ways of doing this, of course, 


but that is their system -- and so, as a consequence, 


in looking at these questions you have to carefully read it, 
and sometimes the language is vague in itself, so that you 
may misinterpret it, and though you may know the answer, 
you might select a wrong alternative. 

For example, take Item No. 2 here, one of those. 


It says: "Of the following the situation in which a super- 


visor would have to make the greatest effort in order to 


on 


~ 
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communicate effectively with his suburdinates would occur 

when", and the correct answer is "(a) There is a large 

gap between the supervisor's backgreu7d and experience and 
that of his subordinates.” 

Well, what is a gap? A gap between what background 
and experience? 

Now, I would say you could make a good case for 
(d) "a supervisor has been with the organization for a much 
shorter time than most of his subordinates." 

He would also have a great comnunications problem, 
and having taken a look at the item tally, which I saw 
later, that 35 per cent of those scoring at the top did 
choose (d) as the answer, and they were marked wrong. 

THE COURT: Why ¢!d you choose (a) instead of 
(d)? 

THE WITNESS: Why did I? 

Because I felt that was the point of the question 
that he was looking for. 

But if I were to review that on appeal I would 
give credit for (d) too. I would say both are equally 
correct. 

Q Would you say (a) is what might be called the 
textbook answer? 


A Yes, I think that is what the current literature 
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says; this is what we are teaching about supervision today. 
I am sure that the constructor of this has a textbook and 
a page number for this question. They are generally 
required to do that, to have an authority for each of these, 
though oftentimes they say many management experts. You 
have to substantiate that, what ig am. sement expert and how 
many is many. 

THE COURT: That is one of the things that I 
dont quite understand, Doctor. I do, of course, agree 
with the implicit assumption both in Mr. Mintz' questions 
to this effect and what you just said, that it is desirable 
that the introductory statement of these questions should be 
true. For obvious reasons it is desirable that it should 
be true or authoritatively stated. But with regard to 
the question of whether or not the answers are appropriately 
related to the kind of work you are doing, and so on, 
doesit really make much difference whether you take a 
hypothetical statement or one that is not necessarily 
authoritatively supported, as distinct from -- well, I mean, 
if you were to say “assume that," would that be so bad? 

THE WITNESS: No, that would not be so bad 
except that someone would challenge your answer, and so if 
you were to just leave out that business of assuming, or, 


as they do, they generally precede the question with a state- 
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ment which not all test constructors do, but they do that 
because they want to be able to cite, to defend that item 
in case it is challenged. 

But it is as a consequence of that that I am 
maintaining that this puts an unnecessary burden on the 
examinee in trying to respond to that and a very knowledge- 
able person, for example, may be confused becaus he knows 
a lot about the research in this particular area, and it is 
not that clearcut. 

For example, the question I missed is the one in 
which I have written two books, and I have taught courses 
for eight years -- 

THE COURT: That happens with me in crossword 
puzzles. I can never get the legal ones. 

THE WITNESS: But No. 15, on interviewing, there 
is no right answer there, becausg first of all, they don't 
tell you what kind of an interview, and you have to define 
the interview, and it depends upon the type of interview, 
because, again looking at the tally percentages, you would 
expect that you would get a pretty even distribution across 


all four answers, and, as I recall, that is approximately 


wlat was obtained. 


So this is my criticism, I would say, of the test. 


And when you have a number of weak items, the discrimin- 
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ability of the test and its validity is going to go down 
considerably. It lessens its reliability because it 
introduces a large error effect. 

If you have a large error effect, then you can't 
get validty of any kind. 

So that is my criticism there. 

Secondly, of course, the test is attempting to 
measure knowledge of; it never measures skill in. 

We addressed ourselves to this question somewhat 
in previous testimony. 

Taking those first 25 questions which I assume 
are designed to measure "supervisory skill,” well, aside 
from the practical question of even defining what supervisory 


Skill is as a construct, defining it, the problem of measuring 


it in a written test, I don't think it has been solved 
adequately. 

I have conducted a number of studies of highly 
standardized tests of supervision developed by National 
Test Publishers, and I have never been able to validate 
it. 

As a matter of fact, it seems to me the higher score 
you get on these tests, the poorer supervisor you are. 


THE COURT: May I ask you this: If you had the 


responsibility of making up a list of people who were 
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supervisors or wanted to be supervisors, how would you 
test them other than by a written test? 

THE WITNESS: Well, you would use -- I would use 
different methods. I have used and de use other nethods. 
Dr. Williams mentioned that one would be 2 
simulation. One method is what they call the group 
interaction test, or group oral performance test where 

they are actually placed in a leadership situation. 

THE COURT: For a certain period of time? 

THE WITNESS: Yes. 

This is being done in the federal government now 
and in industry, they have developed what they call seereeient 
centers, and they bring people in and they put them through 
2-1/2 days of assessment in various situational exercises, 
and from that you can infer whether a person has these 
qualities or not. 

THE COURT: It is quite a job to try to do it with 
regard to 1100 people, isn't it? 


THE WITNESS: Not impossible, though. It just 


recuires a number of examiners. It is the numbers. It just 
means you need money, that's all. 
THE COURT: Yes. That is a problem that I get 


acquainted with from time to time. 
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BY MR. MINTZ: 

Q Well, let us focus on that problem: You are 
hiring people through this test for a job that pays in the 
neighborhood of $15,000 a year ~-- the salaries are in 
evidence, I don't recall exactly what they are -- and 
people usualy have the job for several years, and you are 
investing a substantial amount of money in this person, 
in your experience is there a value to spending more money 
in your selection procedure, maybe even hiring fewer people 
because of it, but getting, therefore, better judgment as 
to who is most competent to do the job? 

A Yes, and it makes good business sense. 

I am about’ to put something like 5,000 managers 
through a aprocedure next year, and the company feels it 
is a very, very worthwhile investment to do this, not only 
because of the salary but the effect on this organization 


and the management of this organization which would ensue | 


from that. 

And interestingly enough, performance on such 
exercise as that does not correlate very well with performance 
On one of these paper and pencil exercises. 

THE COURT: Do sentence completion tests have 
any value? 


THE WITNESS: That is a clinical technique that 


% 
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is used mostly in individual assessment, in personality 
evaluation. But I would not recommend it in a competitive 
selection system. 

THE COURT: Thank you. 

BY MR. MINTZ: 

Q You said also in addition to the first 25 questions 
which were in a field in which you had particular expert- 
ise, you also focused on the reading comprehension and, 

I believe, the math questions. 

Did you have any particular judgment as to them, 
the reading comprehension? 

A Well, the reading comprehension test, I don't 
know it was introduced, because the whole test is a measure 
of reading comprehension, and I am sure that if you took 
those items and correlated them with test performance there 
would be a high correlation. 

You will also notice they did not discriminate 
very much, and one of them was a vocabulary question, not 
really a reading couprehension question at all. 

Q In other words, most people got that right? 

oS Yes, most people got that right, but if they were 
going to test vocabulary, you have got to do it with more 
than one word. I mean, if you just give a person one word, 


there is not much you can infer from that. 
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Q Is what you are saying, then, that if a person 
could adequately read the other 75 questions in order to answe 
them correctly, you are not learning anything more about 
him by asking him five questions to interpret what is in 
the paragraph? 

A That is right. Unless you know for certain that 
reading comprehension is an important component of: the job 
and has a real effect upon job performance, then I wouldn't 
us? it at all if you're measuring it by written test. 

I know it is a favorite device of test constructors 
because they are easy to construct, and th._ fill out the 
test for you, but actually I don't think, without having 
seen the data -- my experience has been that it does not 
contribute much to the variance of the test. 

Q There has been testimony here that reading comp’'e- 
hension is important to this job. 


A Well, there has been testimony as to that 


but there has been no data offered to show that. For example, 


it is a question of reading comprehension and reading speed, 


and what level of reading -- we went into that yesterday -- 
that there are ways of measuring the content that is 

required and I know in another context, for example, there 
are a lot of reading improvement courses that presumably will 


increase your efficiency. 
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None of the studies that I have conducted have 
shown that improvement in reading comprehension contributes 
anything to job performance. 

Q Assuming that testimony that reading comprehension 
is important to this job could be borne out by a real 
measurement, would your juccment about the reading compre- 
hension questions still be the same? 

THE WITNESS: Wonld you repeat that question? 

Q Well, you stated that at least one of your ob- 
ject.ons to the reading comprehension questions is the lack 
of any real demonstration that reading comprehension is 
essential to this job. I am focusing on a particular 
reading comprehension question which was five items on this 
exam. 

Would you sti.l consider that a necessary item 
if it could be proven? 

A In practice what I do, if we analyze it and we find 
that reading comprehension as a function of the job is 
relevant and important, then we can assume that we are 
Measuring that with the test itself. 

THE COURT: At least in this test? 
THE WITNESS: Yes, this type of test, making certain 


that the reading comprehension or the reading level in this 


is the same as in the job, then that would be the measure 


e 
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; of it 
; Q You heard Dr. Katzell's testimony yesterday about there 
' being ways in which you can measure the difficulty of 
. written material that one reads on the job and the difficulty 
° of written material one might put into a test. 
’ Would that be a recognized procedure to make the 
. test more or less difficult to read than the material | 
: a person has to read on this job? 
* A Yes, that is exactly what we do. I think | 
se Dr. Katzell is referring to the Flesch Formula, Rudolph | 
a Flesch, who is another Columbia graduate, and Professor 
| 

” Irving Lord, who really did the pioneer work in this field, 
* but we measure that, and we make certain, and we say, | 
is it the level, if you will excuse me, of the Daily News, | 
” or is it the level of a publication like a scientific 
. journal, or something like that, and you can measure it | 
” precisely. 
7 Q You also, I believe, had some particular conclusions 
* about the math and graph questions here. | 
7 A Well, I felt -- and this is an opinion; I don't 
7 think it is a very important one - but I felt that it was 
- really not a measure of ability to read an interpret 

graphs. It seemed to me it was more a measure of arithmetic, 
25 


‘ at least as I took it, and also reading comprehension. 
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Q You say. through yorr study of this material in 
conversations with me that a number of people who were performing 
the job in fact were unable to pass this test. 

Does this knowledge suggest anything to you about 

the validity of the examination? 

A Yes. As a matter of fact, it was the first question 
that I asked you, because, you see, I think that so far what 
we have been talking about is content validity, and in my | 
opinion all this test possesses -- ¢1d I haven't heard | 
this phrase used so far -- that really what we have here 
is face validity, that it looks like it is about the job; 
and you can do that in many ways, give it face validity, 
but that is not the same as content validity by any means. 
You just put it in the language - you test in the language 
of the job, and it is a good thing to do. It eases the 
applicant's mind somewhat, but it doesn't contribute much | 
to the predictive power of the test. 

Now, the second question - and the way in which we 

begin to validate a test is by givingit to peopie who are | 


performing the job effective, and in my system we then es- 


tablish the cutoff point as the lowest score achieved by 
somebody who was perfccming effectively, because by 


definition that person has the skills and the abilities 


and the adaptability to perform the job successfully. 
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So that when you come down to this question that 
here are people who for two years have been performing 
effectively, and they fail the test; in other words, they 
didn't get 70 per cent - and that is an arbitrary figure; 
that is just picked out of thin air and has no meaning, 
except perhaps in law, but it has no meaning -- but then to 
say that these people are failures on the job is utter non- 
sense. 

So that you would say not that they are inz equate 
performers but that there is something wrong with my test. 

THE COURT: Doctor, yesterday I think it was 
Dr. Katzell, but, anyway, one of the defense witnesses said 
that they did not in their opinion regard the fact that 
you didn't get 70 on this test as meaning that you are un- 
satisfactory on the job; it was simply, as I understand it, 
that those who were doing the hiring, namely, the Civil 
Service Commission, found enough people with 70 and above, 
and they were required by law to take those at the top 
of the heap. 

So, theoretically, you could have had, I suppose, 
out of the 1100 people who took this exam, there may have 
been possibly 800, 900 or 1,000 who could do the jOb, 
but there weren't 800, 900 or 1,000 positions. 


Isn't that a meaningful possibility? 
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THE WITNESS: Well, if you're saying now that the 
purpose of the test was to determine not only whether a 
person was qualified or not but who was better qualified -- 
THE COURT: That is what I understood the defense 
witnesses to be telling me. 
HE WITNESS: Well, that is an unusual procedure 
here,particularly in -- I am not here to discuss the 


equities of that situation, because in common personnel 


practice a person who has been on the job for two years, 
you have a moral obligation to that person, and if they 
established that they are qualified they should get 
preference over any outsiders, and I think the City “oes 
buy that concept because of promotion examinations -- 

THE COURT: Yes, I think in one of Mr. Mintz' 
questions yesterday, it was not challenged that if anybody 
had gotten 70 on th: promotional exam they would have 
outclassed anybody who got a higher yrade on the open 
competitive exam; is that right? 


MR. MINTZ: Yes. 


THE WITNESS: But that is not what they are saying. 
They are saying if you are below 70 you are not qualified. 
THE COURT: I understand what you are saying 
now too. 


THE WITNESS: And that is where the error is. 


25 


slbr Lopez-direct 615 

So, therefore, you see, you don't need -- 

Dr. Katzell referred to the distribution of performance 
rating, to do what we call a product moment correlation 
where you assume that the variables are both normally 
distributed. You can do many other kinds of analyses 
and come up with validities that are based on what we call 
non-parametric methods, just a business of pass, fail, or 
successful or unsuccessful on the job, and we use these 
me*hods in validating tests in private industry. 

So I would say that the very fact that they are 
now contending that a number of people who have been per- 
forming successfully on the job will -- well, you have 
either got to say that there is a mass conspiracy among the 
supervisors to keep a lot of unqualified people in office, 
or we have got. to conclude that the test as it was admin- 
istered is invalid on a criterion basis. 

Yo me there is no other conclusion. 

THE COURT: If they were honestly rated before, 
then there was something wrong with the test? 

THE WITNESS: Yes, honestly. In other words, 


I would find it hard to believe that so many -- in a single 


case, if it is a single case, I can understand, but if 


it is so many, I would find it hard to believe, and I would 


say that a private employer would have a hard time convincing 


to 
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a court that that was so, that these people were not per- 
forming, particularly when you had the qtion of dismissing 
them for incompetency at any time with . any recourse or 
redress. 

So therefore I feel that is the most damaging 
thing of all, the fact that so many people performing this 
job were disqualified. I think it was very bad personnel 
practice too. 

Q Your final conclusion stated at the beginning of 
your testimony was that there was no case for content 
validity in connection with this examination. 

Is there anything further which you feel 
buttresses that conclusion in addition to the specific 
things you said about the job analysis and the exam 
itself? 

A No, I can't think of anything else. I think 
I have presented my perception of the situation. 

MR.MINTZ: I would like to have these two docu- 
ments marked. 

The first is a field audit report, and the test 
plan for the position of human resources specialist, 
dated February 4, 1972, prepared by Mr. Rosenberg. 


The second is field audit report and test plan 


for the position of senior human resources specialist, 


XXX 
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same date, same preparer. 
(Plaintiffs' Exhibits 8 and 9 were marked for 
identification.) 


BY MR. MINTZ: 


Q I believe you have seen those, at least briefly? 
A Yes. 
Q I refer you to the second page on Exhibit 8, | 


and it continues over to the third page where there is a list | 


of qualities purported to be needed to perform those two 


jobs. 
DO you see the point I am referring to? | 
A Yes. The knowledge and skills required ? Is that | 
it? | 
Q Yes, exactly. | 
A Okay. 
Q Do you notice any similarity between the knowledge 


and skills listed there and those listed in Defendants' 
Exhibit F, which is the field audit report on the examination | 
here in issue? 

A Well, they are virtually identical. 

Q I refer you to the last page of Plaintiffs' 
Exhibit 9, the test plan for Senior Human Resources 
Specialist. 


I would ask you to ampare that with the test plan 


~ 
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which is Defendants' Exhibit G in evidence, and which you 
also have, which we had sent to you previously for the test 


here in issue, the supervising test. 


A I have it, yes. 

Q Do you notice any similarity there? 

A Yes. They are virtually identically worded. 
Q How about the weights? 

A The weights vary to some extent, slightly; 


5 per cent. 

Q Isn't it a fact the only variance, inclusion of 
one 5 per vent item on office machines and related 
factors on the supervising test which -- 

A No. l, functions of HRA -- oh, between supervising 


and senior? 


Q Yes. 
A Oh, I was looking at the lower level job too, 
specialist. 


Yes, you're right, the only varianc2 between the 
two is on machine equipment, 5 per cent, and arithmetic 
computations is 15 per cent on senior and 10 per cent on the 
supervising. 

Q If you were a person asked to develop two tests, 


based on those two tests plans would you have any way 


of developing two different tests? 
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A Only to a very minor degree. You know, I believe 
this machine equipment and supply, I might put a few items 
in there that I wouldn't include on the senior human 
resources specialist, and I probably would reduce the 
amount of arithmetic computations and interpreation of 
tables and charts on the supervising. 

Q Otherwise they would be identical? 

A The same tect because the job analysis said these 
functions are identical. 

THE COURT: But if the job analysis is correct it 
wouldn't be surprising that the tests would be almost 
identical, is that correct? 

THE WITNESS: Yes. 

THE COURT: The question is whether the job 
analysis is correct. 

THE WITNESS: It is quite possible the two jobs, 


different levels, in some areas could require the same 


| 
traits. You see, that happens all the time. It is something 
| 


else that accounts for the difference in level and the 
complexity of the position. 

Q Would there be any way on the basis of those test 
plans on which a test constructor could determine that an 
easier or harder question should be put on one exam or 


the other? 


to 
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A No, there is no basis for doing that. That 


would be injecting the test constructors own opinions into 


the test plan. 

There is also, as I pointed out to you earlier, 
in this test plan a great deal of latitude for the test 
constructor. 


Q Dr. Lopez, would you be able to come to any sort 


of nclusion with any reasonable degree of certainty about 


what this test does, in fact, measure? 

A Well, I would assume -- first of all, it measures 
reading comprehension at some level, and fairly abstract 
at that. 

Secondly, it does show familiarity with 
literature on shite practices and some behavioral 
science concepts touching on management and supervision. 

It also measures knowledge of certain. topics 
and agencies, and so on, that would deal with the work of 
a human resources specialist. 

Q Would there be any basis,in your opinion, from 
your examination of these materials and familiarity with 
the facts here for determining whether someone who had 


this knowledge to enable them to correctly answer those 


questions would, in fact, be better able to do the job 


at issue? 
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A Well, based on my experiences with this type 
of examination, I find that it will contribute very little 
to the prediction of performance in the jobs for which it is 
being used. We find that this type of test rarely 
correlates very highly with any index of performance. 

Q Is there any other purpose psychometrically or 
from a personnel expert's point of view for giving an 
employment test other than to try and predict as best you 
can effective performance on the job? 

A No, there is not any other reason for doing it. 

Q Finally, there is in evidence here, I am sure 
you have heard, that people who took this test had widely 
varying educational backgrounds, from high school 
equivalencies and, in some cases, perhaps lower, to graduate 
degrees. 

Do you feel that formal education would affect 
performance on this examination in any way? 

A Well, I would assume that a person who is college 
trained would have had much more exposure to tests of this 
sort, and would be more test wise, and therefore, they would 
perform better on this test than those who had not had much 
exposure to testing, which is one of the things we have 
learned in schools, how to take tests. 


But other than that, I don't think that the person's 
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background experience would have much effect on this. 
Q What do you mean by test wise? Is that a 


professional term? 


A Yes. 
Q What does it mean? 
A Well, there are two tasks when you take a test, 


and one is learniny how to take the test; in other iain: 
following instructions, the type of thing Dr. Williams 

was referring to yesterday and this mroning as part of the 
content validity. 

If you give complicated instructions that may be 
the real test, just to find out how to take the test; and 
sowhen you repeat a test, you give a test to a person two 
or three times, their score usually improves because 
they know how to take the test. 

So in taking multiple choice tests in particular, 
there are a lot of rules you can follow, and a person 
who has learned those rules, that system, can many times 
get a fairly impressive score on a test about which he 
knows very little. 

MR. MINTZ: I would like to introduce Plaintiffs' 
Exhibits 8 and 9 into evidence and that will conclude my 


direct examination. 


THE COURT: I think there is no objection? 
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MS. OMANSKY: No. 
THE COURT: All right, Plaintiffs' Exhibits 8 and 9 
for identification are received in evicence. 
(Plaintiffs' Exhibits 8 and 9 for identification 
were received in evidence.) 
CROSS EXAMINATION 


BY MS. OMANS..Y: 


Q Dr. Lopez, do you know Dr. Richar ! Barrett? 

A Yes. 

Q Would you regard hin as an expert in the field? 
A Yes. 

Q W.2n was the last time you worked for the City 


of New York? 
A I think it was January or December 1973. 
Q December 1973? 
A Yes. 
THE COURT: January or December 1973? 
THE WITNESS: No, December 1972 or January 1973, 
right in that period. 
Q Did you get paid? 
A Yes. 
Q And the time that you worked for the City and 
accepted money, were you as unhappy with the city's 


personnel policy? 


to 
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Yes. 
But you did the work nevertheless? 
Yes. 
We will get into that in a moment. 
Did you pass an examination for the diplomate? 
Yes. 
What kind of an examination was it? 
Oral. 
It was an oral examination? 
Yes. 
Entirely oral? 
Pardon me? 
was it entirely oral? 
Yes. They waived the written. 
They waived the written for you? 
Yes. 
Why did they waive the written for you? 


Well, I think they we:re waiving it for a number 


of people who had exter ive experience. 


Q 


A 


Sort of a grandfather clause? 


No, not grandfather. The grandfather diplomates 


were automatically given a diplomate. 


Q 


A 


You had to take an oral? 


I had to submit a research project that I had done, 


~ 
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and then I took an oral. 


Q For people for whom the requirements were not 


waived, did they take a written test? 


A Some had, yes. Yes, if it were not waived. 
Q They would take a written test? 
A They did. I am not sure whether they still do. 


There was a lot of unhappiness with the written test. 


Q You are not sure that they still do? 


A No, I am not sure whether they dropped the written 


test. They may still require tiose who are very young 
with little experienr to take the test. 

Q These people who are very young with little 
experience who were required to take a written test, 
what would that test be testing for? 

A They would test knowledge of psychology, 
principies of psychology. 

Q What traits does it test? 


A Knowledge -- 


Q Knowledge is a trait? 

A No, no -~ knowledge is a trait -- 

Q That is what I said, knuwledge is a trait? 

A Yes, so they would be testing craft knowledge, 


know.cige of various information that would be require‘ 


to perform as a professional psychologist. 


a 
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Q This would be knowledge of the principles of 
psychology, I suppose? 

A Yes. 

Q Would knowledge of other areas be a trait that 
you would test for? 

A Well, knowledge -- we are talking about a 
trait that is known as craft knowledge. That is one 


human trait, the possession of certain knowledge that 


you have. 
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Is qaaft knowledge restricted to a profes- 


Sional or does it apply to everybody? 


A No, no, knowledge of his craft, you see. 
Q His craft? 
A Your craft knowledge would he different, 
you see, Yours would be knowledge of law. 
Q Right. 
A Okay. So if you want to say that is a dif- 


ferent trait, you can, but I am generalizing and 
calling all these craft knowledqe, and we require vou to 
specify what that knowledge iz that you must have, 
¢) I assumed that the knowledge you require as 
a lawyer is different from the knowledge you require as 


a psychologist? 


A Right. 

Q It’s knowledge in the area, right? 

A Yes. 

Q Craft knowiedge doesn't add anything to your 
explanation, does it? I mean is it reasonable to test 
for -- to test me on principals of psychology? In 


other words, when you are giving a test for knowledge 
to add the word “craft” knowledge doesn't really add 


anything, does it? 


A It specifies thetype of knowledge we are look- 


ive) 
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ing for, you see, hecause knowledge can he general. 


Itincludes tlie whole universe, 


( I suppose most tests that test for Iknowledge 


test for craft knowledge? 


Av {O's 
() 102 
aN IO. 


) ~O doctors -- 


N ‘ow you say if it is a test to predict per- 


formance in an occupation, that's right, but there 


are many tests of Fnglish literature and many other 


things. 


Q Is that a test of knowledge? 


It is a test of knowledge but not craft 


knowledge. 


“HE COURT: Unless you are an Mnglish pro- 
fessor. 
THE WIIESS: Yes. 
©) Wien did you agree to testify in this case? 
A Pardon me? 
Q When did you agree to testify for the plaintiffs 
in this case? 


A I am trying to recall. I S“elieve it was in 


Octoher,. Hecause then I forgot about it. 


625 


mda 3 Lopez-cross 

Q bid you agree when Mr. itintz called you up 
and asked you if you would testify? bid you agree 
then? 

A Yes. He called me and asked if I would 
testify in this case, and I asked him who it was. 2 


should have perhaps elaborated on this a little bit this 
morning, because I have often been called to testify 
and I never accept a case for a plaintiff in a civil 
service examination because I have only testified for 
the city, but I never testified for a plaintiff »ecause 
I don't believe in the system of taking -- of appealing 
a test to court except in a civil rights matter. 

So in this particular instance I wavered a 
Lit but J felt that this was an opportunity to make a 
constructive contribution to the field of public person- 
nel selection. 

Q Would it be fair to say your only considera- 
tion when lr. “Mintz called you on the telephone was basically 
this was a civil rights case and it wasn't against the -- 
the only qualification you mentioned, tnat was your 
only consideration? 


A Well, I assumed that the situation, the issue 


would be one with which I am very familiar and have 


discussed at length with a great many people in the field, »pt 
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I alwavs reserve the right, in reviewing the testimony, 
or in reviewing the facts in the case, which I didn't 


get to see until the middle of December, or see suffi- 


cient facts -- I always reserve the right not to testify jf 


THE COURT: May I ask -=- maybe Ms. Omansky 
is going to, but I don't want to take a chance on it -- 
what you thought might be your contribution to civic 
policy by your testimony here? 

THE WITNESS: Well --’ 

THE COURT: Or what contribution you would 
like to make. 

SHE WITNESS: In my discussions with a great 
many people in the field of public personnel they are aware 
as I am of tue inadequacies of the system and they are 
aware of the fact that it is having an adverse impact on 
minority groups, but they are locked into the system and 
they have said, “We can't do anything about it hecause 
we can't get the money to change the law and to do these 
things." 

So as in private business they said they 
couldn't afford to do it, but then the courts began to 
issue some very severe sanctions andthey got the message 


and they found the money and they are now developing sound 


I felt in fact the city had a good, strong case. 
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2 selection procesures, and that is all that the guidelines 
3 and the law require any employer to do, is to maintain > 
4 a sound selection systen. 
5 Q So when you agreed to testify you didn't have 
6 the job analysis, you didn't have the test plan, is that 
7 right? 
8 A That's right. 
9 THE COURT: I understand you agreed to 
10 testify sort of sight unseen with, however, you said, the 
11 reservation that if after you saw you didn't agree, then 
12 | you would withdraw. 
| 13 THE WITNESS: That's right. nis I have a. 
14 been surprised before in cases when you finally see the 
| 15 material. In the opposite way I once felt there 
16 was an employer who I felt was perfect, and when I began 
17 to see what he had I said, "I couldn't testify for you." 
18 BY MS. OMANSKY: 
19 Q Are you getting paid for this testimony? 
20 A Well, JT hope so. 
21 Q How much are you being paid? 
22 A I think the fee is $259 a day. a 
23 Q Is the job analysis, a document known as a 
2A job analysis, a summary of a procedure? I mean is that - 
25 A Is a job analysis a summary of a procedure? 
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{) Yes. 

A No, the job analysis is the procedure. The 
final documentation, you have many pieces of paper and 


data in the final job analysisthat supports the selection 


strategy that you employ. 


Q You have outlined several steps of that? 
A Yes. 
Q I assume that the job analysis, the piece of 


paper that emerges, is the culmination of your procedure? 
A It is the final selection plan in my system 
that comes out. 
Q Would it have been relevant to know in more 
detail how the job analysis was made in this case than 


you had when you agreed to testify? 


A Yes, it would have if I had had much more 
evidence, But I understand none was submitted. 

Q You have the depositions, right? 

A I have the deposition 

Q But that was much later, after you agreed to 
testify? 

A Yes. Well, yes, as a matter of fact, I 


forgot all about it because I get calls from NAACP would 
I tastify in Atlanta, Georgia, and I never hear from 


them again. This one, as Mr. Mintz snows, I had even 
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scheduled sonething else for yesterday and today. I 

had to move them to he ahle to he here. But it was 

all based on the supposition on my part that what I 

would find is what I would expect to find, and if I found 

something different I naturally wouldn't have testified. 
Q You are pretty available to the NAACP Legal 


Nefense Yund, are you, as an expert? 


A No, I have never testified for them before, 
Q) But they have called you hefore? 

A Yes. 

Q And you agreed to testify subject to your 


qualification if you found something that disappointed 
your preconceptions you would not? 

A Yes. 

Q Would you say your preconceptions at all 


influenced your judgment in this case? 


A No. 
Q How did you discount your preconceptions? 
A Because I take a look at the data submitted 


and I look for certain requirements in it that meet 
certain professional stanuards. 
Q Your preconceptions in no way influence you? 


A How could they? Here it is. The fact is 


a fact. Here isthe data. One cf the first questions 
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IT asked was what was the adverse impact. If there, 
for example, had been no adverse impact there would have 
been no case, 

() If there had been no adverse impact with the 
oxamination would it have been job related? 

A No, it wouldn't, but then you could do anything 
vou like. You could use tea leaves to predict if it 


has no adverse impact on anyone, 


HL COURTS You mean legally you can? 

THE: WITTESS ¢ Yes, legally. 

SHE COURT s Has anybody thought of that? 

THE WITNESS s They have handwriting 
analysis and a few other things like that. But the 


point is many of the employeers heretofore in their 

selection procedures were doing no more than that and 

they weren't contested, and he was at perfect liberty 

to do this unless he had an adverse social impact on a 

particular group asnd then it's up to him to show that it 

is related to performance on the job. 

Q So they tell me. I understand that burden. 

You said if there was no adverse impact you 

would be free to do whatever you want. Are you saying 

nonminority persons wouldn't have a legitimate claim 


if the examination was not job-related? 
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THE COURT: I think that pr. Lopez is saying 
only what he understands the job to be, that jolb-related- 
nes3 is irrelevant, tliat nobody looks at it if the 
examination doesn't have a particular kind of impact, 

Is that what you mean? 
THE WITNESS Yes. 

BY “S. OMANSKY: 

Q Supposing a nonminority person said, "Gee, 
I don't think this exan is job-related." Would the 
lav look at that? 

A Iam not a lawyer. 

Q I thought you were telling me about the results 
of adverse impact and substantial burdens, 

THE COURT: I don't see the necessity of going 
into this. Mr. Lopez may be right or wrong about the 
law, but maybe he has no competence to tell us what he 
thinks the law is. 

To clear up any misconception, obviously anybody 
has a right to complain but they have to demonstrate an 


impact, that is what you are saying? 


THE WITNESS: Yes. 
THE COURT? And then the question of job- 
relatedness becomes relevant, I think all “ir. Lopez 


is saying as he understands the law -- 
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THE WITUESS: I am also trying to say I 

wouldn't get involved in the case if it were just 

contesting an examination saying it is irrelevant, you 

SEA. I would only get involved in a case on the hasis 

of the fact that it had also an adverse social consequence. 
THE COURTs In other vords, you are an 


acknowledged crusader? 


THD WITHESS : All right. 
THE COURT: I don't say that in any pejora- 
tive sense. You have a particular interest in this 


kind of testimony and only this kind of testimony. 
Tie WITNESS: That's right. hat’s right. 
BY MS. OTUNNSKY: 

Q) I suppose if I came toyou and said I just took 
an examination for attorney in the Law bepartment of the 
City of New York and I think that examination was not 
job-related you wouldn't be interested in testifying for 
me. 

A You see, I am not a professional witness. “Ay 
yusiness is I am so busy installing programs and admini- 
stering them that it's a financial sacrifice for me 
to take the time to come down here and to testify in this 


case, but I feel that i ou want to call me a crusader 


you may. 
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THE COUR": You feel it is worth while? 
THE WITNESS: Right. 
Q It wov’.dn't be worth w’ ile for me? You 


wouldn't do it for me? 


HE COURT? He didn't say that. He would 


do it for you if the impact was unfavorahle for your 
particular group. 


Is that right? 


THE WITHESS: Yes. 
Q If I were a minority? 
THE COURT: Not just if you felt unhappy 
about it, 
MS. OMANSKY: I appreciate your attempts 


to speak for Dr. Lopez, but I don't quite understand. If 
his interest is in reforming the civil service system, 
that surely that interest shou 1 extend to reaching | 
good examinations for everyone. 

THE WITNESS: That is my ultimate purpose, 


nut I haven't got the time, and I don't think it would 


have the impact hecause majority groups have been appeal- 
ing examinations for the last 40, ) years, and what they 

have been doing is really weaking the system rather than 

strengthening it, in my opinion. 


But I feel that we have a chance with the 
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minority group question now to force the authorities, 

the legislature. principally, to adopt enabling legislation 
that will permit the Civil Service Commission and the 
Personnel Department to do a proper jolh. 


BY MNS. OMANSKYs 


() Lots of money too, I suppose? 
A Yes. They have said already it will cost 
| 
a billion dollars to do it. You knov that the 


International Personnel ‘!anagement Association has re- 
viewed the most recent edition of tie -- the proposed guide- 
lines, the uniform guidelines, and are opposed to it. 
They are holding it up right now because they say they 
don't have the money and they couldn't possibly conform 
to them, 

THE COURT: A billion dollars for what? 
For the whole country? 

THE WITNESS: Yes, for the whole country. 
They say the federal government should provide the funds 
to do this. 

Q Will you tell me about the "'uman Resources 

organization? When was it organized. When did you 
set it up, incorporate it? 


A My company 


Q Yes, your company. 
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A I thought you meant -- 
THE COURT: What is it called. 
THE WITNESS: Yelix '!. Lopez Associates, 
Inc. 
THE COURT: Where did you get the name? 
THE WITNESS: I looked around and wane 
to name it after the finest psychologist I knew. I 


established it on January 1, 1970. 


Q How many employees are there? 
A Five. 
Q How many analyses has the human resources 


organization done? 


A Over 209. - 
0 You have done over 299 in three years, right? 
A Yes. We don't do them, you know. The 


client woes them under our direction. 

THE COURT: What would you say, you supervise 
them? 

THE WITNESS: We teach them the system and 
provide them with the tools and train their people to 
do it because they have to continue to do it from here on 
out. 

Q So you train outside organizations. When you 


say -- when I ask2d you the question how r.. iy job analyses 
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have you done the answer is none, isn't it? 


A No, no, because I sign every one of them, 
a) You sign? | 
A Yes. | 


.@) And do you go out and -=- how do you know whe- | 
ther the job analysis was done properly? 

PA Well, you would have to see the systen, ut 
it is very easy to see whether it is done properly be- 
cause there are a lot of checks involved in it that we 
can make, 

Q Could you name some of the positions for which 
you have done job analyses? 

A Yes. Secretary, salesman, retail sales 
clerk, store manager, catalog district manager, vice 


president, merchandising, power plant operator, lineman, 


cciaputer programer, The wnole gamut. 

THE COURT: Could you name some of your bet- 
ter xnown clients? 

THI WITNESS: Yes. The Federal Reserve 


Bank, the Pord Foundation, Montgomery Ward, American 
Electric Power, 

Q As I understand it, you go to the Ford Founda- 
tion or you have people from the Ford Poundation coming 
to you, you give them a bunch of materials -- 


A NO. 


| 
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Q How do you do it? 

A We go to the client. 

Q What do you do there? 

A We put them through a training program that 
lasts maybe two to three days to learn the system, We 
do a lot of things before that. We first review 


their whole job structure and we develop what we 
call a career plan matrix. We develop jo! families 


and levels. 


Q How do you do that? 

A Well, we take the levels hy their salary 
structure. 

Q Sort of -- 

A Yes, it's approximately. We don't have, 
you know, we don't develop minute ones. We have a 
broad category. We have one category level unskilled 


or semiskilled, then we might have skilled, and then we 
might have senior or supervisory. 
In management we have junior management, 
middle management and top management, three levels. 
Q How long does it take you to do this? 
A It depends on the size of the organization. 
A company like Montgomery Ward, with 125,000 employees, 


it takes six months of study. They find out they have 
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a mess. You see, this is the first thing you 

have to do, unravel the organization, if you are going to 
be ahle to develop a selection plan for them. 

Then we pick a key job to analyze. We don't 
analyze every job. A key job is one to which a reason- 
able number of people are expected to move within a 
reasonable period of time. Of course, that is now going 


to be defined in the uniform quidelines. 


So that is the job this person is going to 


move to. We analyze that joh. And then we -~ 
Q You pick one key job? 
nr Pardon me? 
Q) You pick one key job, right? 
BA Oe In the job family we may for good 
reasons pick three cr four. It varies. 
Q What is the shortest period of time that it 


took you to develop this career matrix? 


A It took about tvo hours in the Ford Foundation. 

Q So you develop this career ix, and then 
you what? 

A Then we identify the key jobs we are going to 


analyze, because that was what I was starting to tell you, 
we analyze those jobs -- 


Q Who is we? 
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A We, the client and the consultant. 
Q You and your five employees? 
THE COURT: And the client. 
A NO, generally just one of us does this. tt 


doesn't take -- because we use all of the client's per- 
sonnel from the personnel department, the management 
structure, »ecause the idea is to develop the system and 
install it and they operate it from then on so they have 
to learn it, and this requires a completely different type 
of thinking, a different approach, to selecting. “ost 


people think in terms of requirements of five years of 


experience or a level of educétion,. We eliminate 
those. 

Q What was the company with the 125,000 employees? 

A Nontgomery Ward. 

Q How much did you charge them? 

A iow much did we charge them? 

Q Yes. 

A We are still doing it, you see. 

Q How much did you charge them so far? 

A I don’t know. I can't tell you exactly. 

Q How many tests -- well -- all right, I will 
go on. 


How many tests have you conducted? 


6, 
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A How many tests? 
"HE COURT: You meanssince he has been in 


private practice. 


(2) Yes, in the human resources organization. 
nN - We have constructed and validated criterion- 
related validity now, I'd say, ahout 30 tests, 30 or ° 


0) Nid you dothe tests yourself or was it again 


a >rocess of instructing? 


A No, we construct the tests. 
Q For what positions? 
A We don't develop tests for positions,you 
3ee. We develop tests to measure your constructs. 
Q I see. 
A Or traits, as we refer to them. But then if 


the job analysis shows that that trait at that level is 
required for successfui job performance we then use -- 
we say that test that measures that trait is the one that 


ost be used in this selection process. 


THE COURT: bo you sometimes take part of me 
test and <.rt of another? 
HE WITHESS: No,because the test has already 


been invalidated on a group of occupations, you see, on 


current employees, and the test is invalidated as a whole 


as an entity, so you don't break it up. 


Sy 
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THE COURT: I think “is. Omansky is about to 
ask you something about the tests, so go ahead, 
BY MS.OMANSKY: 
Q What traits are these you testified to? 
A One vet of perception-memory-concentration. 
They are very carefully defined, 


Another one would he -- 


.@) How (oO you test for perception-memory 
concentration? 

A By using items that would requ.ce a ,:rson's 
perception. One wey is a comparison of fie res or 
numbers. That 1s one ofthe types of items used. 

Q Is it a written test? 

A Yes. Yes, I am not against written tests. 

Q Are all the tests you have constructed written 
tests? 

A No. We also do performance exercises. We 
don't call them tests. We have tests of machine opera- 


tion, equipment, stenography, typing. 
Q Tell me some cther traits that you hav con- 
structed tests for? 
A Decision-making, problem-solving. 
Q Wait a minute. Don't go so fast. 
Tell me how you test for decision-making. 


A You use essentially a simulation -- one of the 
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metnods we use is an in-basket exercise, originally 
developed hy r7S. They taught me how to do that. 

THE COURT: I don't know what that is. Can 
you tell me? 

THE WITNESS: That is an exercise where you 
put a person in a situation where you say You just moved 
into this job, your predecessor has left it, he has 
left behind this in ‘basket with all the correspondence 
and letters in it. Behave as if you are on the joh. 
‘lake decisions on each of the items. Then we evaluate. 

It was Originally designed by the chief of 
research at I'S, Dr. Norman Fredericksen, and pr. John 
Hemphill. 

Q Tell me how you test for problem-solving. 
A You use items similar to that. You can give 
them a situation or a diagram, something like that, and 


create a problem for them, 


0 See A simulation? 
A It's a simulation, yes. All of these tests 
are based on actual work content. They are material that 


is used on the job in the job into which the examinee is 


expected to be placed. 


Q How much do you charge for constructing a 


trait test? 
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B We don't charge on that basis. We have hourly 


rates, you see, that depends upon who is conducting 


the test. 
Q What are your hourly rates? 
A Mine are about -- 
MRe MINTZ: I object to it as heing 
irrelevant here. I don't think what he charges for what 


has any relevance to his opinions in this case. 
MS. OMANSKYs I think it will have a great 
deal of relevance as I develop it, frankly. 


THE COURT: NO you want to Let the cat cut 


of the hag? 

MS. OMNANSKYs I want to know how much it 
would cost to do -- he has moved ayay from the answer 
as to how much he charges to construct a test and I want 
to do a “%b analysis and it is important to know the 


cost of these things. Certainly I will point out it is 


a relevant factor. 

THE COURT: I don't see any reason to go 
into confidential information unless it is necessary, but 
I think you can ask the witness -- if I thought it were 
necessary I would ask for an answer, but I think you can 
certainly ask Dr. Lopez what it would cost to do certain 


things. 
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A I will respond to your question, I know 
that is what you are leading up to. I don't like to 
give out information because our charges to clients 
vary, and -- 


THE COURT: That is why I didn't want 


to press you. 
A But I will tell you you would be amazed how 
cheaply this can be done. 
7 THE COURT: Dr. lopez, I don't want to ask 
you what you charge certain clients, and I am reluctant to 
allow Ms. Omansky to do that, because I don't think it 


is necessary for our purposes. If it were, I would. 


On the other hand, can you give us some 
feel of what the scope of cost of these exercises is? 


THE WITNESS: At the request of the Depart- 


ment of Transportation I served as an adviser to them in 
a study leading to the validation of one test for bus 
driver across the country doneby the University of Chicago 
and the feces in that case for one test was -- one selec- 
tion program, just for one job, bus driver, was about 
$325,000, ; 
THE COURT: Are you talking about the total 


cost? 


THE WITNESS: I restructured the whole 
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“ontgomery Ward System from top to hottom for less than 
that. 

BY "1S. OMALISKYs 

Q For less than the cost to do one test for one 
bus driver? 

A They were using what I call very -- and they 
didn't have a job analysis, but I am saying if you are 
thinking along those lines then you are going to get a 
billion dollar figure. But if you think in terms of 
what it can be done for -- 

Q You mean how much you could do it for? 

A Not only I. But that can he done for -- 

Zz would say the Montgomery Ward, when it is all finished, 
will have cost probably no more than £250,900 over a 
four-year period. 

THE COURT That is the total cost to the 
company involved? 

THE WITNESS: Yes, for 125,000 employees, 
It is less than $2 an employee. 

Q I am going to ask another cost question. You 
said that you did a fireman's study some Place for battalion 


chief? 


A Yes, in another municipality not far from New 


York. 
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\) What municipality was it? 

A Philadelphia. 

0 How much did that cost Philadelphia? 

A $6009, 

Q Yor the position of hattalion chief? 

\ And assistant fire chief and deputy fire 
chief. There were three jobs that we tested for. 

Q iiow many jobs, to your knowledge, and you are 
pretty familiar with New York City practice -- how many 
positions does New York City test for each year? 

A How many positions? 

Q Yes. 

A Or jobs? 

2 I will take positions first, and you can distin- 
guish for me, 

A You see, a job i3 a class of positions. A 
position would be filled by a single body. 

Q How many jobs then? 

A Jobs is what you are talking about. 

Q Right. 

A How many do you do a year, test for? 

Q Yes. 

A Are you talking about competitive examina- 
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are a lot of examinations conduct! where there is no 
announcement because I sat on orals for those many times, 
But just those where y «e@ publishing announcements I 
would probably figure you are Going maybe 1500 a year 
or so, 20990, 

Q Would cost be a relevant factor to the City 
of tiew York for 1509? 

A Of course itwould be relevant. 

v Your last contact with the City of New York 
was in December of 1972 or January of 1973? 

A That was the last time I worked for them. 


Q Did you ever construct any examinations for the 


City of New York? 


A Yes. 

Q For what jobs? 

A I can't remember them all, they are in my files, 
but I did a whole personnel examining series. That 
is principal, senior, personnel examiner. In all 


probability Mr. Rosenberg took that examination. 


Q For principal and senior? 
A Yes. 
THE COURT: That means he is well qualified 


if he passed. 


THE WITNESS: Wait a minute now -- 


mda 26 Lopez-cross 652 


\) You constructed the examination? 

A I participated in it. 

@) Now many other people participated? 

A The one I am thinking of there was one other 
person besides myself. We did one part of it, and 


Bg 
'ducational Testing Service did the other~perts~ And 


tren I served asthe appeals for manifest errors for their 
part. 

Q Did you make a job analysis of principal 
personnel examiner? 

A No. We were given a test plan. 

Q) You were gi 1 something similar to what you 
have here? 

A Yes. In other words, you see, they made the 
decision what the test would cover. They had already 
done that. 

Q Was that satisfactory to you? 

A I am trying to think back. It was satis- 
factory, yes. I think so. 

THE COURT: You mean the test plan they gave 
you, the particular test pla: they gave you was satis- 
factory, or the method was satisfactory? 

THE WITNESS: No, the ,»articular test plan 


was satisfactory at that time. You see, this was maybe 
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fovr years ago. I am not talking about the most 
recent ones. All I did, the most recent one, was I sat 
on an oral hoard for some agency for personnel examiner, 


and then, you know, we pass them and that's the end of 


it. 
Q But four years ago it was satisfactory? f 
A Professor Sidney Mailick was with me then. 
THE COURT: I am still not clear. If 


it were January, 1973 would th‘s test plan -- would you 
have considered it satisfactory, or was it just four 
years ago you were satisfied to proceed in this way? 

THE WITNESS: No, four years ago I was. 

I wouldn't say I was overjoy2d with it but, like everybody 
else, I said we will do the best with what we have. This 
is the law and so -- 

THRE COURT: So you are not saying they gave 
you a good or bad test plan, but that you were satisfied 
with the way it was done. 

THE WTINES< : Yes, and for my part I was to 
develop the content for the written. It was an essay 
tes::, not a multiple choice, and I was to cover the content 
of a personnel job. There were a number of specialties. 


They were examining people, testing people, training 


people. 
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Q Where were all different examinations? 

A No, it was one examination. 

Q What traits did you determine were necessary 
for personnel? 


a I didn't do that. All I was doing was 


testing knowledge. 


Q Is that a satisfactory thing to be tested for? 
A In a personnel person? 

Q | Yes. 

A Yes. That is one of the requirements. 

Q Did you happen to ask in the test that you 


prepared for personnel examiner ~- was one of the questions 
how do you prepare a job analysis? 

A I can't recall. It may have been, There 
was something to do with job analyris in the test. 


Q Would it be relevant to ask a candidate for 


the position of personnel exéminer how to prepare a job 


analvsis? 
A Yes. At the level that we are looking for. 
Q nid you know whether there were persons taking 


that test who did not prepare a job analysis? 


A Yeas, 
Q Were there? 
A Yes, there were. 
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@) Was it job related to ask them how to do a 
job analysis? 

A If they were promoted they vould get on the 
list for personnel examiner, senior or principal per~ 
sonnel examiner, I am not sure which now, and they could 
fil] any vacancy that came up. 

Q Would that be desirable, to test somebody 
for a job that he doesn't do right now »ut which he might 
be called upon to do? 

A Well, it was more than that. We weren't 
testing for what he might do. We felt that reyardless 
of whether you were in training, classification, examin- 
ing, there is certain knowledge about these other 
disciplines or the subdisciplines that you should 
possess, A good personnel man in the section field 
knows classificatin and compensation. 

Q What else would he know? He would know 
how to do a job analysis, I suppose? 

A No, he would have to know the purposes of a 
job analysis, and also be familiar with the various ways 
of conducting it, the various plans, things that are in 
any elementary textbook on personnel administration. 

Q Things that are in any elementary texthook 


you would expect them to know, I suppose? 
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A Yes, of course. 

Q Why? 

A Why? 

Q Why would you expect tiem to knowthings that 


are in an elementary textbook? 


A Why would I expect them to know that? 
Q Yes. 
A Because he would have a difficult time per- 


hasbeen functions of a perconnel technician, a per- 
sonnel examiner, without being aware of these things. 
That's the trouble with a great many people in the per- 
sonnel field; they really don't know personnel. 

Q So you would want to test for the knowledges 
that the, sould -- 


A I would expect them to have certain basic 
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knowledge, be aware of certain literature on the subject, 
and have done some reading in these areas. 
Q Would you expect the same thing of a supervising 


human resources specialist? 


A Yes, I would. In his field, not personnel. 

Q Certainly. Personnel is, I admit, a Pena care 
field. 

A I don't have any problems with the generic 
position, I listened to it. 
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Q It is proper to test for a generic position? 
A Yes, it makes good sense to do it that way? 
Q Let's go to the Port Authority. What jobs 


were you responsible for testing for in the Port Author- 


ity? 
A All of them, 
Q Would you name them? I only work for the 
city. 
THE COURT: He named a lot in his earlier 
testimony. 
A We have maintenance operations and clerical 


and administrative are the three broad areas in what 
we call the servicee. That is the nonexempt cateyory 
of positions, 

We most commonly selected people in the 
maintenance field at the laborer's level, the entrance 
level,and would test them for that. Then there was 


a series of competitive promotional examinations all the 


way up through the automobile mechanics series, electrical 


maintenance, the plumbing, stearfitting and so forth, a 


watch engineer and s0 on. 


Then «» the clerical side you had clerk i, 2, 
3 and 4, 


Q Almost like the city's. 
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A Yese 

THE COURT: What was the largest number of 
employees there when you were there? 

THE WITNESS: About 8900. 

BY "IS .OMANSKYs 

@) How did vou test for maintenance operation, 
clerical and secretar’ 11? 

A It varied, depending upon the job. In 
some jobs we -- for example, in clerical there would 
always be a written tes! an! an oral. In maintenance 
tnere would be more lii.c?y a performance test and an 
oral. We did not yive a written test except for cer- 
tain jobs where, for example. you had to know electrical 
theory, and the best way sf measuring your knowledge of 
electvsical theory is a written test. 

But we almosi: always developed performance <-- 
standardized performance exercises where the knowledge 
of how to fit a joint or how to wire something was im- 
portant. We found that there was a big difference 
between those who could do a written test and those who 
did a ii aida exercise. 

Q What traits were you testing for when you 
were testing? 


Py I don't know. You are taking me back now to 
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before I developed this system. All I was running into 


were the problems you were running into. 


Q It wasn't under civil service, was it? 
va No. “ 
sat 
Q You had an o,x0rtunity to do that on Ye su Own, 
right? 

A Yes, I did. 

Q And you weren't }hound by civil se-«-rice? Vy 
THE COURT: I gather God ‘ia. not illuminated 


your mind to that extent? 

THE WITNESS: No, it took a lot of time, 
This is the way things develop. You run into a serious 
problem and you begin to study it more intensively and 
you say here is tiie cause of my problem, and so I felt 
that the real problem was the inadequacy of the job 
analysis that we were doing, because I first ran into 
this problem in the '50s in the Port Authority. 

Q By the time you left had you instituted you: és 

four-step system of doing a job analysis by first doing 
an industrial examiner who makes a microscopic analysis 
and then you have a systems analyst who makes a flow 
analysis? Did you incorporate that system into the 


Port Authority? 


A No. I }new of it ther but what we didn't 


have was the force. I had been interested and I had 
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been following the work of a person named Primoff who 
was a psychologist with the Federal Civil Service 
Commission, and his J coefficient notion, and it was 
somet ' I wanted to work on. One of the reasons I 
left the Port Authority was to have the opportunity to 
pursue these things, because within an organizatonal 
context you don’: have the freedom to do these things. 
THE COURT: Especially the Port Authority. 
HE WITNESS : “here was & great deal of free- 
dom to innovate but this has changed. 
THE COURT: Let's relax for 10 minutes. 
(Recess, ) 
BY MS. CMANSKY: 

Q What did you do in the City of Philadelphia 
for the hattalion chief and the two other positions you 
mentioned? What were they? 

A Battalion fire chief and then deputy fire 
chief and assistant fire chief. The assistant is 
next to the commissioner. Their civil service law 
requires that all of those positions be executive, unlie 
New York, where they are appointed. They wanted an 
examination and they had a lot of problems with the 
multiple cheice format as had New York City, so they 
asked me to come down anduse different techniques similar 


to what I had used in the frelection of police lieutenants 
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and captains in the Port Authority. That was another 

position, by the way, please, How could I forget them? 
Q Did you do a job analysis? 


A Yes, that is what I described this morning in 


my testimony. 


Q liow long did you do it? 
A It took one of my associates -- he spent 
about three days down there. Then with the material 


we collected on the questionnaires, the filling out 
of the PAQ, we sent that to Purdue University where it 
was put through the iii aia we got a printout back 
with their traits. 

Q What traits did you identify? 

A I should have brought the report with me because 
I can't recallthe names of the traits that he used in his 
system. They are a little more elaborate than mine, 
but it is in the literature, He has published widely. 


All psychologists are familiar with his work, or nearly 


all, 
THE COURT; With whose work? 
THE WITNESS: With McCormick's work. 
Q Did you prepare tests for these people? 
A Yes, then we constructed a test. 
Q What sort of test was it? 
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a An in-hbasket test. 
Q That was a simulated situation? 
A Yes. 


THE COURT: You said suppose there was a 
fire,what do you doe, or something like that? 

THE WITNESS: Yes. Interestingly, we 
found in many respects there were no differences in the 
job, in requirements. 

Q No difference in the job in requirements or 
what? 

A Between these three levels of top supervision 
in the Fire Department. 

THE COURT? Battalion chief and assistant 
and deputy. 

A The battalion chief, then the deputy and then 
the assistant. For example, on decision making, 
which was one of the traits that comes out of this 
detailed analysis, you get it in reference to many 
other occupations, and so bikes found i:at it was 


identical for these three levels, so therefore we were 


justified in administering the same test to those three. 


Q Would it be poasible to have the same knowledge 


necessary for a senior human resources specialist as for 


a supex *t{ng human resources specialist? 
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(on) 


‘A It could be possible, yes. I said that. 


That is possible, 


2 That is possible? 
A Yes. 
Q So the fact there is an identity of knowl- 2 


edges and skills doesn't say anything? 

A iio, that doesn't say anything. But unless 
every aspect of the job is the same, and then you would 
question why you have three levels. 

Q You said you tested, you did an in-asket 
test, for deputy fire chief. iow many people took 
this test? 

A About 100, 125. You see, you had to be a 
fire captain to take it. 

THE COURT: You are talking about deputy or 
batcalion? 

THE WITNESS: There were some hattalion 
chiefs taking it to complete for the higher level position. 
But altogether there were about 125. That is my 
memory now. 

Q In tlew York City, as I well know, in the Fire 
Department. there are about 10,000 men, and the officer rank 
is fairly small, and I would assume that most of the fire- 


men, the entry level position, compete for the fire 
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licutenant's position, which is the next position. 
Would it be possible to give an in-basket test or a simu- 
lated-situation test to 8009 people? 

A It would be possible, bu” / jon’t know 


whether you would do it or not. 


Q ¥ rouldn't you do it? 

A B.° .use J would have to see the analysis. 
Excuse me now. I am not familiar with the Fire 
Neeneuiuihs but I am with the Po.ice Department, In the 


Police Department I know there are deputy inspectors and 
inspectors and then borou7) commanders and so on. There 
are only 160 of them because I conducted a course for i 
them. There are only 160 of then. Philadelphia 
is a smaller city. 

Q Would the ability to make decisions be 


relevant to a fire lieutenant in New York City? 


A I would think so. 

Q How would you test for that if you had 8000 
candidates? 

A You are getting free consulting here. 

Q We need it. You say we need it, anyhow. 

A You do, 


I would rather say how I did it for the 


Port Authurity police lieutenants. 
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Q You are quite an experienced tester, you 
have your own firm and have done a lot of work. Based 
on your expert opinion I would like to know,if the 
ability to make a decision is important to a fire 
lieutenant, how you would test for that if you had 8000 


people taking the test? 


A We are assuming now it is important, right? 

Q Yes. I assure it is important. 

A It is also what kind of uecisions do they make 
and in what contextare those decisions made. If they 


are administrative decisions ~- 

Q Let's take decisions how to fight a fire. 
You have to make a decision on how to fight a fire. 
That ig a fairly skilled art, isn't it? 

A Yes, it is. 

Q I assume that every time a truck goes out 
there is a decision that is made as to how to fight that 
fire. 

A I don't know if it is so. I think it is 
standardized and he follows the regulatiomand procedures. 
He might get into a situation where he can't follow regula- 
tions or procedures and this would call for a decision 

Q You think that might occur often in a fire 


situation, which is unpredictable? 
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A Not often. No, I don't think it occurs 
often. It does occur, though, and that is the critical 
time, 

Q It is critical, isn't it, the ability to make 


a decision? 
A Yas, and you should measure it. 


Q vant to know how you measure it if you 


have 8090 people. 


iN rf you have 8009 people? 

Q Yes. 

A I think it can he done, I don't know just 
how. I would have to give some thought and study to 


what kind of a simulation I would set up where I could 
also administer it to 8999 people. That is not a de- 
terrant, incidentally, just sheer numbers. 

THE COURT? Explain why it ‘s not. 

THE WITNESS: It is just a matter of hiring 
people if you need them to administer. 

THE COURT: Even, for example, if you only 
have 50 to conduct this test, if there were 8900 people, 
you might do it over a longer period of time? 

THE WITNESS: That's right. We do this. 


The argument, you know, that we know this procedure is 


very bad but it is cheap and it is convenient to me 
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just doesn't stiund up. We have got to find other 
ways of doing it, and I think we can do it, and I think 


the city has the competence to do it too. 


OQ How would you test for tolerance? 
Pa Tolerance? 
9 Yes. 
A Not all of uu. « traits can be measured 
by a wr-icten test or a u_mulation,. In some instances 


we would have to use verification procedures to inquire 
into their background or previous experience. Tolerance 
you might. Some people try to do this with stress 
interviewing: put you in a situation and see if I can get 
youto lose your temper. But those are unreliable, 
and they also are bad public relations. 

Q In the Philadelphia case you say you 


distributed scuestionnaires? 


A Yes. 
Q To whom did you distribute the questionnaires? 
A We distrivuted them to a number of assistant 


fire commissioners, to the fire commissioner, to some 
battalion chiefs and so on. 
Q Ye sked them what they did, is that correct? 
A The questionnaire has a great many questions 


on it. They go into various aspects of it, not simply 
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into just what you do by any means, He goes into the 
degree he will describe something, and say is it better 
in this position sometimes, always or never. 

It is probably the most widely known of the 
worker~oriented job analyses and the research is still on- 
going. 

Q Do you have any union p~oblems when you distri- 
buted questionnaires to people in the Fire Department in 
Philadelphia? Did the union come around and say, "Is 
this going to affect anything in our collective bargain- 
ing agreement" etc.? 

A Ho, it didn’t. Elsewhere where I have done 
other types of job analysis we involve the union in the 
job analysis and they are very cooperative. 

Q I suppose if you submit these questionnaires 
to people you presuppose everybody is more or less 
doing the same job, is that correct? 

A You don't presuppose it. You select people 
as being <~-presentative as far as you can see of the 
posit.ions tat exist in that organization. It is one 
job vut we have varying positions in it. 

Take a fire lieutenant. There are some 
fire lieutenants doing fire prevention work and they are 


not out there in the firehouse at all. So that job is a 
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totally different job. But it is a city test for one 
job. 

Q And it is okay to test for one job? 

A NO~. 

Q Did you give different tests? 

A I would. ts ae job analysis calls for a 


different set of characteristics on the part of the in- 


cumbent you would then have to use different measures. 


Q How many tests did you give for hattalion 
chief? 
A Weonly gave one. 
Q How many tests did you give for deputy fire 
chief? « 
A One® 
Q And for assistant fire chief? 
A One. 
Q Did you determine to your satisfaction that 
all assistant fire chiefs were doing the same job? 


A Yes, 

Q It would make a difference if everybody was 
doing -- 

A I saia chis morning if we find differences we 


test -- one of them is what we call the interrater 


agreement. You see, you can get a measure there, and 
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if you have a wide discrepancy in their perceptions of 
the job you don't just average it, you see, you go 

back and say, “Ilow come? Fither one of you filled it 
out inaccurately or you are describing different jobs.” 
That is the way in which you prevent a selection error 
later on down the road. 

So we find thatthe statistics show there was 
an interrater agreement something in the 30s, which is 
very sites 

THE COURT: What's the word? 

THE WITNESS? Interrater. It is simply 
a correlation between the ratings. 

A If you are using the instruments, that is the 
first thing you have to determine. 

Q To the best of your knowledge of city experi- 
ence, New York City and other iaicinia Nati would it make 
any difference if you submitted these questionnaires to 
a group of provisionals? 

A It wouldn't make any difference. If they 
are performing the job and the requirement that you have 
is that the person knows the job, that's all. I don't 
care what his status is, if he really knows that job, 
and you have common sense reasons for supposing he does 


have it, then it is important to do this. 
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VHE COURT: If you had been commissioned 
to handle this situation here in New York would you 
have administered a questionnaire to a certain number 
of people, for example? 


“HE WITNESS: I would have done <-- yes, I 


would have done in analysis that would have heen, you knov, 


particularly faced with the situation that was as I 
understand here, where you have people with one title 
hut doing -- working in different agencies and with sub- 
titles, parentheses, parenthetical titles, I would check 
that out very carefully, because frequently you do find 
that the jobs are different, they call for different 
skills, knowledge and ability. 

THE COURT: Putting aside the question of 
differences or similarities, would you have been 
concerned that administering the questionnaire, if that 
is the right terminology, to some people who might be 
taking the test might give them a break over others who 
were taking the test to whom you did not administer the 
questionnaire? 

THE WITNESS: I wouldn't have any -- I don't 
see how that could possibly he. 

THE COURT: You don't think that the tone 


or approach of the questions need necessarily give any 
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indication of what it is likely to get on the exam? 
THE WITNESS: No, I don't think so at all. 

You might fird, as we did frequently in the Port 
Autharity, that we would have part 1 of the test 
that would be for everybody, and then part 2 would be the 
parentheses, and the candidate had the option of 
selecting the one he wanted to take. So that might 
have been a desirable procedure in this case. 
BY “iS. oO LANSKY : 

Q You mentioned that you would take a 10 per 
cent sample, 

A That is a generally accepted sample. 

Q If you had 100 people surveyed in a job, I 
guess that would be 10 people? 

A Yes. But that would be a minimum, They 


would have to represent a cross section. 


Q You would have to know the job? 
A Sure. 
Q tow long did it take to accumulate the 20,000 


items in your file? 


A Probably 10 or 11 years. 
Q Do the items ever get out of date? 
A Yes. 


Q How did you determine this? 
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A When they were chosen for -- you see, we had 
a system there, The technician could use the file 
but he had to buy the questions by submitting five new 
ones. That is one of the ways we kept it current. 
Of course we would Only use a few of these because they 
were essentially untested questions in the test plan, 
and that is the way you would == then he would look over 
items in a category and he would see a question that 
no longer applied. 

We also, incidentally, something that I am not 

sure you do in the city, there was a time when you did 
not do it, is we always submit the test to the sub ect 
matter experts in the department. If it were a police 
test we would have top people in the Police Department 
take the test and criticize it for any technical inade- 
quacy or aberrations in it. 

Q If you give the test, if you try the test out, 
if you pretest it, do you have any problems of security? 

A Yes. You have to do it in a way that it will 
remain secure. 

Q How do you determine whether the test items 
remain secure? 

A The way we did it was it was either we tested 


it out in some other area that was not related to this 
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or we had top level people, you know, take the test. 

Q Supposing you were testing for craft knowledge. 
Could you test it out in a different area? 

A NO, no. We had a different philosophy there, 
but you had to maintain the security of the test, but if 
you tested craft knowledge that was absolutely essential 
to your job,if a person kne. this we didn't feel that even 
knoving it in advance gave him any advantage, but we 
maintained the security. 

Particularly this is true with performance 
tyne tests, you see. In other words, you don't have 
to keep it a secret somebody is going to have to type a 
letter, or a carpenter would know that he is going to 
nave to do hammering. 

Q I suppose it is a lot easier if you are giving 
a performance-type test? 

A Yes. 

Q If you are going to be able to use items again 
I suppose that the job itself has to remain fairly stable? 
I mean you would have to make sure the job hasn't changed. 

A Yes, and know because you are vsing items 
now. You don’t use the same tests again. You can 
keep using the items if they are measuring a certain, 


particular area that is critical for performance. This 
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isn’t that important. Sometimes you would drop some- 
thing out. 

For example, take a bus operator. At one 
time he had to know some simple arithmetic. But then 


you went into the locked=-box system and he doesn't even 


need to know how to make change. 


Oo Did you give the bus driver a written test? 
A The bus driver? 

Q Yes. 

A Yes, I did. “his was not in the Port 


Authority, this was in ‘!4BSTOA, a subsidiary of the New 
York City Transit Authority not under civil service. 
I vas invited to participate, to conduct a series of 
examinations for hus driver to dispatcher, and I did, 

Q Is the ability to read important in being a 
bus driver? 

A NOe 

Q I thought you testified before that the whole 
test more or less is a reading comprehension test? 


THE COURT: What I got from Dr. Lopez before 


was that a test could be a reading comprehension test by 
setting it at the level of the reading required by the 


job when reading is required by the job. Is that 


right? 
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THE WITNESS: That's right. The only 


reason you would eparate it is if varying degrees of 


proficiency in reading were important in the job. 


Q 


candidates. 


What degrees of proficiency were important 


in reading for the bus driver? 


A None. I said that. 

.) cheless you gave them a written test? 

A Yes. 

0 Did you test a lot of bus drivers? 

A Yes. 

Q Is that one of the reasons you gave a written 
test? 

A I didn't test the bus driver. It was for 
promotion to dispatcher. The bus drivers were the 


I introduced into the city the concept 


of eliminating all tests for bus drivers. I said the 
Only test you need is the possession of a driver's license, 
and on that basis for the first time there was introduced 
into MABSTOA a large number of minority groups. I 
checked on it later, and to the horror of the super- 
intendents, depending on what is called the APA examina-~ 
tion they had heen using, we found in the accident there 
was no change, there were no other changes in the perfornm- 


ance of the system. 


® FF B BRB 


677 


mda51 Lopez-cross 
Q Did you do the same thing for dispatchers? 
A No, for dispatcher the strategy we used here 


was we developed a written test based on knowledge of 
geography because the dispatcher had to tell people where 
to go in the city, he had to make up schedules, and we 
had that in there. But, you see, we don't use an 
arbitrary pass point. We cut off at a level that we 
felt that was so low that there would be -- that if the 
person dien't pass that he probebly couldn't perforn 
in the job effectively, or at least not as well as 
the others, and the rest of them we then put through an 
oral simulation exercise that was tape recorded. It 
was guite interesting. 

AS a consequence of that, the objective that 
I was after was achieved. On the eligible list were 
as many -- the same proportion of whites that were in the 
African population, and the same proportion of 
blacks, and so as a consequence of this had we set a 70 
per cent cutoff we would have screened out a great many 
of the blacks, but as a consequence of this a _~eat many 
of them were promoted to the job, and as a result of that 
we were sued by the whites. 


Q Problems never end, do they? 


Did you ever hear of Delehanty‘ s? 
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A Yes. 

Q What did they used to do hefore they went 
bankrupt? 

A I knew Mr. Deleharty. He ran a school ot 


teach you how to pass a civil service examination. 
He would get the book and drill you in it until you 
passed. 

THL COURT: A. cram course? 

THE WITessSs: Yes. He branched out into 
a high school later, 

Q What if Delehanty's or an analogous organiza- 
tion got some of the questions and prepped people on it, 
and tiew York City, using an item bank, used those ques- 
tions. Do you think there would be an objection to 
the exam? 

A They would have to develop quite a massive and 
expensive program to be able to offset this. They 
tried to do it with me in the Port Authority. 


In New York City for a period you did not 


release examinations, you were attempting to go in this 
direction, and the candidates would go in with the 
agreement they each would memorize one question and they 


would come out and put the test together, and within 24 


hours they have your test anyway. That was done in 
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the Port Authority, but that didn't have much of an impact 
our construction of the tests and so forth. 
Q Was it a factor? 
A I could never identify it as a factor. I 
don't think we ever had a complaint on that score, 
I might say in the City of Philadelphia in 
the tvo series I ran there there were no appeals. 
THE COURT: You mean appeals were possible 
but none were taken? 
THE WITNESS: Yes. ‘My contention is 
when an examination is relevant and logical you do reduce 
the appeals to almost nothing. 
BY 4S. OMANSKY: 
Q llow Many appeals were there in the supervising 


human resources specialist exam? 


A I don't know, 

Q Suppose I told you there were 10 appeals? 
A I am glad to hear it. 

Q If there were no appeals would it be your 


position that that exam was probably a pretty good exam? 


A In one respect, in terms of candidate accept~ 
ability. 
Q When you were testifying you said you testified 


in New York State courts, is that correct? 
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A Yes. 
Q And you were testifying to uphold -- may I 
suggest this was an Acosta-type challenge, that the 


candidate's answer was as good as or better than the 


examiner's answer? 


A I testified for the Corporation Counsel. 

Q That's r'jht. 

A On seven questions dealing with supervision 
and manayenent, and I was the expert. They were being 


contested hy those that were <= it was a fire c: -tains 
list, and only on those questions, and all the questions 
vere upheld by the judge. 

Q Did you honestly believe that the questions 
that you were upholding were job-related questions? 


A I wasn't te.cafying on that. I was testify- 
ing on the accuracy of the key answer. 

Q Being as good as or better? 

A It wasthe best answer and the only answer I 


testified to. 


Q bo you suppose somebody could do the same thing 


for the supervising human resources specialist test, testify 


that the answers were the best answers and the only. 


answers? 


A In some of those questions I would testify to 


— 
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that. Not all ot them. But then somebody else 


Q You said it is very difficult to measure your 


supervisory skills, right? 


A Yes, it is. 

Q And that you yourself had never heen able to 
validate? 

A two widely published tests of supervision. 
One is "How Supervised," — i’ .ished by Psychological 


Corporation, and the other one is -- I fo; get the nane 

of it, "Principles of Supervision," by Martin bruce, who 
is a test publisher up in Westchester. I just 

finished this big study of that one involving 500 managers 
with zero correlation. 

Q Wher you were testing in the Department of 
Personnel were you testing for a supervisory position? 

A Yes. I also did some supervisory administra- 
tive positions outside of the Department of Personnel for 
the Department of Personnel. 

Yes, supervision was involved. 
Q Were thosequestions valid? 
A I didn't do the questions on supervision. 


ATS did those. 


Q Were ATS questions on supervision valid, do 
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you know? 


A Yes, I think it was. They used an in-bavket 


exercise, and there was an appeal from that, and I was 


the technical consultant on the appeal. Being one of 


the few people that understood in-hasket exercise, they 


nad no choice, outside of ETS, 
Q You talked about these assessment centers, 


and you said you are now putting 50900 managers through? 


~ 


A We plan to do that. 


ra) Horr long would it take for each candidate to 
go through this assessment center? 


A There are “ying models. The model that 


I have finally devised -- well, I haven't finally de- 


vised one, ‘ut I hope to one day. The telephone company 


put them through in two and a half days. They only 
do that once in their career, 

Q Once in their career? 

A Yes. The Federal Civil Service is using an 


assessment, but I don't know what model they are going to 


be using. They are also interested in it. 
Q The Federal Civil Service also uses reading 
comprehension? 


A I don't know what they are doing. All I 


know is some of the agencies are going to use the 
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innovation in selection, 


Q 


Do you know whether the Federal Civil Service 


entrance examination test is a reading comprehension test? 


sustained 


A Yes, it is, It is in litigation, 

Q Do you know whether or not it has heen 
so far? 

A It has heen sustained hut not by the Supreme 
Court. 

Q What is content validity? 

A Content validity? 

Q Yes, 

A 


It is the test or the instrument measuring the 


trait or the factor that is associated, 


that it purports 


to measure, I think I would Say it that way. 
Q Is the trait a factor of knowledge or a 
personality factor? 


A It could be anything, It could be a skill 


such as typing, or it could be driving an autombile, 
and it could be a trait, Sometimes when we use it as 
a trait we call it construct validity, 


Q Is construct validity the same as content 


validity? 


A Yes, pretty much, 
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Q If you want to test how much someone knows or 


has learned about a field wat would the appropriate 


kind of test e7 


A T£ you want to test pure knowledge? 

Q Yes. 

A Well, if he can read and vrite I would use a 
paper-ar.iji-pencil test. Otherwise I could use an oral 
test. I have done that many times. 

Q Would it »‘e a content validity or a construct? 

A That would >be a content validity test becuse 
it would 


“how the knowledge you are using in your test is 
also used on the job. But you see the point is in 
terms of your total selection strategy, if you just hase 
the selection on that you might be in serious error. 

Q You mentioned something about mass conspiracy 
among supervisors, You were very much disgurbed by the 
fact that supervisors in this case rated, you think -- 


by the way, who did you speak to in this case besides “xr. 


Mintz? 
A NO, one. 
Q No one else? 


THE COURT: I didn't get from Dr. Lopez’ 
testimony what you just said. I got from his testimony 


that he believes that if a large number of people pro- 
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visionally in a job didn't pass the test, then either 
the test was poor or one had to assume that the super- 


visors were conspiring to keep people who were unqualified 


in the job in the job. 


Is that right? 


THE WITNESS: Yes. 

THE COURT: You didn't say they were doing 
it. 

THE WITNESS: Yes, that through incompetence 


they were permitting people to function. 
BY MS. OMANSKYs 

Q You did say that the problems of criterion rat- 
ings could be overcome, Does everybody agree with you 
on that? 

A That is « broad question, I don't know 
whether everybody agrees with me on that. If you 
ask if the body of psychologists who have studied and 
addressed themselves to criterion problem, would they agree 
with me on that, I think so. 

Q I don't want to confuse you. I am going 
back a minute to the difference between content and con- 
struct validation. 

Does the American Psychological Pores eer ee 


distinguish between them? 
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A Yese They have four different types, ut 
I think they also say that construct validity is a special 
case of content validity. 

Q Would you say that in test development the level 
of predictors is higher than criterion development? 

A This is my opinion and I would say yes, that -- 
I better say it because I have written it, that psy- 
chologists have addressed themselves more to the predictor 
side of this equation and have neglected the criterion side 
of it. ‘They have just assumed that the criteria that 


they were using were accurate and correct. 


Q What is the difference between Criteria and 
predictors? 
A A crition is z measure of job effectiveness. 


The predictor is simply a measure of a trait in a person 
that will indicate whether or not he will be effective 
in the job. 

Q De you think an aspect of criterion contamina- 
tion is a situe*ion in which a person might rate another 
person high or low becasuse he is of a certain ethnic or 
racial group which he favors or does no- favor? 

A That is not contamination; that is just 


bias. 


Q Do you know if -~ 
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@) DO you know if -- 
A Contamination is where some other influence, 
such as knowledge where a person stands on a test, you 


see, would contaminate. 


Q That is the only aspect of contamination? 
A Chat is the way I use it. 
Q eo you have any idea as to whether or not you 


can overcome the problem of bias, as you call it? 


aA I hope so, 
Q Hias it been overcome, that problem of hias? 
A It*s been minimized, I don't think you can 


ever say -~ essentially you are -- well, at some place 
you can, if you use some cbhiective indices, If you 
useattendance records or pure production records that 
would be an unbiased rating. It may not be the most 
accurate or the best but it is unbiased. But*here you 
are getting into the position of one person evaluating 
the performance of another, it is rather difficult to 
minimize the effects of bias, 

Q How would that affect your judgment as to how 
Supervisors rated provisioncls in this case? 

A You see, I am assuming == and I think it is 


a valid assumption -=- that you have to take as a fact 


that if the supervisors continued to permit these people 
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to perform on the job they are seiiniaiikiid it satisfac- 
torily,. If the work of the agency is going on and these 
people are presumably in fairly key functions, then I would 
gay that de facto they are performing satisfactorily. 

If you are going to say who is performing most 
satisfactorily, this is the question supervisors have more 
difficulty, you see, in distinguishing. They don't 
like to do this. Generally speaking,they want to 
give aiieetaatha pretty much the same rating. 

Q Let's talk about that. If they wanted to 
give everybody the same rating, would that affect your 
judgment as to the validity of the evaluations in 
this case? 

A It is perfectly possible to conduct a validity 
study with just two dimensions to your criterion. One 
is satisfactory-unsatisfactory. You see, you don't need 
to have a normal distribution. If you are going to 
try to identify superior performance or something like 
that,then you would need a greater differentiation in 
your criterion. 

Q If you just had a satisfactory-unsatisfactory 
evaluation, and everybody was satisfactory, is that of 


any value if you are trying to equate their performance 


onan exam with that evaluation? 


— 
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A That is a crucial factor. Here I have all 
acceptable employees. If I cive tiiss test and I 
find that some of the employees have scored 12 right 
on the test of the 75, then I would Say anybody scoring 
above 12 can perform the job satisfactorily. 

9 Did we finish up how you would account for bias 
in our case? 

A That would take me into an extended discussion 
of a technique or techniques that have been developed. 

The one I use is known as the forced-choice technique. 
Dr. Melanie Baer of the University of Chicago prepared 

a comparison technique, and she uses that. She has 
elaborate statistics to show how that has eliminated my~ 
self, particularly if you correct for length of service 
and seniority. 

Q I am asking in this case what weight you gave 
to the factor of bias or possible bias in the Supervisory 
evaluations, I had the impression you said that 
wasn't important, but what was important was that everybody 


was rated satisfactory. 


A I don't think I said either of those things. 
Q Would you clarify? 
A I said Idon't think that a differentiation on 


criteria is essential to validate a test for the decision 
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of pass or not pass, to determine whether they are 
qualified to do the job. If you are in the position 
of selecting the best qualified from a group of people 
you would like to get more, but you get into another 
problem on your criterion side of having what we call 

a skewed distribution. You don't have normal distribu- 
tin. 

You would assume that unsatisfactory people 
or those inarginal employees have been eliminated by the 
selection process or !.y the supervisory process, and so 
you have mainly effective performers and perhaps some 
Superior performers. Nnd so the notion of linear 
progression between the test score and job performanceis 
generally not borne out in real life. SO we are not 
concerned ahout it 

Q In an earlier question I described a situation 


in which you said that wasn't criterion contamination, 


it was bias. Professor Williams says it is criterion 
contamination. Would you disagree with that? 
A That is his terminology in his hook. I have 


a book too, and I call contamination something else. 
Q What do you know about the position of super- 


vising human resources specialist? 


A Very little. 


w 
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Q But you made a statement that the exam was 


not content valid? 


A That's riglit. 

0 On what basis? 

A I said there was no evidence of it. 

Q You didn't speak even to any of the plain- 
tiffs? 

A Right. 

() You spoke only to “ir. ‘fintz or Mr. Howard? 

A Yes, I spoke to ‘ir. Howard too. 

Q Did you think the purpose of this exam was to 


predict performance? 

A Yes. 

Q Would you expect a person who has a master's 
degre in social work to do better on this test than some- 
ody who did not have a master's degree in social work? 

A I don't know. I couldn't make any expecta- 
tion because what does the other person have, you see. 

19) I think my question was a statement on which you | 


would have imagined the better educated would be better? 


A I found people with a limited education who 
have performed hetter in certain jobs than people with 


graduate degrees. 


Q Would somebody who has a master's degree from 
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the Columbia School of Social Work be test-wise, do 


you think? 


A I would guess so, yes, }retty much. 
NS PONANSKY: I have no further questions. 
THE COURT: Any further questions? 
4, MINTZ: No redirect. 
THE COURT: Thank you very much, Lr. Lopez. 


It was quite interesting. 


Are the plaintiffs resting now? 


MR. MINTZ: Yes. 
THE COURT: Are the defendants going to 
call anybody? I am not inviting it. I just want to 


know what the technical position is. 


MS. O:UANSRYs No, your Honor. 


THE COURT: Then can 2 3ee counsel briefly 


in the robing room to seewhat arrangements we can make 


from here? 

(Discussion off the record.) 

(In the robing room.) 

“MR. HOWARD: I discussed with counsel for the 
defendants and they have agreed, with respect to the 
witnesses James Jones and “Mary Eccles and Andrew Jaci.son, 
that they are all African, ‘lack. We thought we would 


put that on the record. 
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THE COURT: If they had not stipulated I 


would have taken judicial notice. 


MR. MINTZ: The Second Circuit might not. 
THE COURT: That's right, I guess. 
MS. ONANSKY: That is why we asked Pr. William 


the question. 

IR. MINTZ: Tie figures I talked about the 
first 15 minutes of the trial, we haven't yet gotten a 
response from the city, and they should be a matter of 
record. 

(Discussion off the record.) 

THE COURT: Counsel for the defendants have 
been instructed within the coming week to arranye for 
submission to the court and to plaintiffs' counsel of a 
resume or a recap of the number of hlacks and whites who 
passed and failed the examinations here, 2913 and 1631, 
the number of blacks and whites presenting serving as 
supervising HRSs, the numbers of blacks and whites who 
would be appointed to the position of supervising HRS if 
the exam results were effectuated. 

THE COURT: In an off-the-record discussion 
it was indicated te the court that the parties have 


agreed that the plaintiffs may and shall by January 18th 


file a proposed amendment to the complaint or amended 
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MR. HOWARD: In the nature of a decision. 
THE COURT: Adding a liispanic plaintiff and 


allegations as to his background. 
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SEcoND AMENDED Notice o¥ Examination NuMBER 1631] 
OFFICIAL 
; Cal. #677 - 9.28.72 
Notice of Examination 56040 
2nd AMELOLD NOTICE 
(October 5, i972) 
No. 1631 


PROMOTION TO SUPERVISING HUMAN RESOURCES SPECIALIST 
Candidates who filed previously need not file again. 


this examination is open to el 


igible employees of all affected agencies of City government. 
A separate eligible list will be 


established for each agency. 
> 2, fi 
SALARY: The minimum appointment rate for th's ition is $32.509 per an 
APPLICATIONS: Received in person only, vctover 5, 6 and 10, 1972 From 9:3u a.m. Lo 4:00 p.m. 
in Room 219, 55 Tnomas Street, .ew York, Py: Ps Voie ) applicet ix j ived by 
mail. 
DATE OF TEST: The written test will be held on October 14, 1972, , 


ELIGIBILITY REQUIREMENTS: Open to each employee of all affected agencies 

of City government who on the date of the written test: (1) is permanently employed 
in the title of Senior Human Resources Specialist; (2) has served as a permanent 
employee in such title for a period of not less than six months immediately 
preceding that date; and (3) is not otherwise ineligible. 


Persons serving in the eligible title and who obtained such title pursuant to Rule 6.1.9 
4i}1 be credited for service in the prior title for meeting th: eligibility requi ents 
for this examination. 


DUTIES AND RESPONSIBILITIES: Under direction, with broad discretion for the use of 
dependent judgicat and initiative, supervises and directs a group or groups oF Senio 
Human Resources ‘Specialists and lower level employees of the ijuman Resources Oc- 
cupational Group’ and/or other employees, in planning, executing and evaluatin 
programs relative to one or more of the following areas: manpover and career development, 
management, community development, or program planning and analysis; or performs complex 
and highly responsible staff assignments in relation to any of the programs listed above; 
performs related work. 


EXAMPLES OF TYPICAL TASKS: Supervises a group or groups of Senior liuman Resources — 
Specialists and lower level employees of the Human Resources Uccunational Group and/or 
other employees; in planning, executing and evaluating any one or several of the above 
stated programs; plans, assigns and reviews the regular and special work of the group and, 
as necessary, trains and instructs members of the group; coordinates all th» activities, 
functions and reports, including research, of assigned group; develops and implements 
Systems to be used, policy procedures and overall plan of operations; assesses and de- 
termines goals and priorities; adopts and translates researci knowledge into workine 
practicalitics; maintains liaison with business, labor and industrial organizations 

and public, private and community organizations and agencies, to negotiate contrac 
renders advice on program.policics, plans and budgets; seeks improvement in faciliti 

and activities; reviews compliance with terms of contracts; may provide professional 
expertise in the area of specialty; interviews and participates in the selection of 
prospective professional or clerical employees, clients and trainees; assists the director 
or assistant director in formulating policy; preparcs summaries of program plans, research 
and evaluation reports for use by the director and/or commissioner before governmental, 
private or civic organizations; designs, develops and implements projects involving 
complex, sensitiv. and far-reaching problems and policies related to the area of specialty 
mects with officials and representatives of private and public azencies dnd community 
groups, to accomplish any of the above, to provide functional services, or to obtain 
feedback as to policies, programs and procedures;may assume the responsibilities of 
immediate supervisor in the absence of that individual. 
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TESTS: Seniority, weight 15; written, weight 35, 70% required; oral, weight 50, 

70% required. The written test will be of the multipie-choice type and may include 
questions covering the following areas as they relate to Supervising Human Resources 
Specialist: Functions of the Human Resources Administration, its constituert 
asencies and other relevant public and private agencies; current developments and 
problems in the field of human services; community relations and organization; 
judgment in on-the-job situations; techniques of staff development, supervision 

and guidance; language usage including vocabulary and reading comprehenston; 
arithmetic computations and interpretation of tables, charts and graphs and other 
related areas. 


The factors in the oral test will be speech, manner and judgment. The oral test 
will be designed to measure ability to respond to stress situations, to handle 
and supervise people, to meet effectively with high administrative officials, to 


conduct conferences,¢to deal with community groups, e.c. 
The pertinent sections of the General Examination Regulations and the General 
Provisions of the Notice of Examination are also part of this Notice. 


CITY CIVIL SERVICE COMMISSION: Harry I. Bronstein, Chairman; 
George Henry, Jr., Secretary. 


October 14, 1972 


SIGNALS 


BOOKLETS 


LEAVING 
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{ Examination NuMBER 1631] 
THE CITY OF NEW YORK rors 
DEPARTMENT OF PERSONNEL Lain 


PROMOTION TO SUPERVISING HUMAN RESOURCES SPECIALIST 
EXAMINATION NO. 1631 


Written Test - Weight 35, 70% Required 


GENERAL INSTRUCTIONS Time Allowed: 


First Signal: Question booklets distributed. READ THIS PAGE ONLY. 


Second Signal: Open your booklet. Begin work. 

Third Signal: End of test. Stop all work. This signal is 
given 245 hours after the second signal. lf 
you finish before th» third signal, summon 
the monitor. No extra time is allowed on 


account of lateness. The time allowed includes 


time for fingerprinting. 


This test booklet should consist of 19 consecutively numbered 
pages. There are 80 questions, all ot equal weight. ANSWER 
ALL QUESTIONS. Examine your booklet after the second signal 
is given to be sure it is not defective in any way. You are 
responsible for obtaining a comple*e booklet. 


You may make a record of your answers in the test booklet and 
you may take the test booklet with you. This is necessary in 
the event that you wish to protest the proposed key answers. 
The time allowed includes time for recording your answers and 
no time may be taken for this purpose after the third signal 
is given. Further information concerning key answers and 
protests appears at the end of the test booklet. 


If you read these instructions carefully, you will not need 
to make inquiries. No monitor is authorized to change these 
instructions or the terms and conditions of this test. You 
will be DISQUALIFIED if you obtain or attempt to obtain in- 
formation or help from books or other sources, copy or at~ 
tempt to copy or read the work of another candidate, permit 
your work to be copied or read by another candidate, com- 
municate in any way with another candidate, leave your seat 
without permission of the monitor, sign your name or put 

any identifying mark on any answer sheet, or disobey an in- 
struction of an examiner. You are prohibited from having on 
your desk or in,your possession during the course of the 
test any reference materials, notes, or newspapers which 

you have brought with you. 


It is a misdemeanor for anyone to impersonate a candidate 
or permit himself to be impersonated at an examination. 


Smoking is NOT permitted. 


During the test you may not leave your seat except when neces~ 
sary, and then only when accompanied by a monitor. Yo.1 may 
withdraw without completing this test and not have your test 
rated by writing "I withdraw" and your signature on your an~ 
swer sheet and giving it to the monitor. towever, no candi- 
date will be permitted to depart before 10:30 a.m. 


DO NOT OPEN THIS BOOKLFT UNTIL THE SECOND SIGNAL IS GIVEN 


No. 
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DIRECTIONS FOR ANSWERING QUESTIONS 
Answer all questions on the official Answer Sheet before the third bell has rung. 
ONLY YOUR ANSWER SHEET WILL BE RATED. 


For each question, select that answer which you believe to be the acceptable one 
among those listed and note the capital letter prececing your selected answer. 


Then, on the official answer sheet, find the row numbered the same as the question 
and record your answer by blackening the space between the dotted lines immediately 
to the right of the capital letter representing your selected answer, using a No. 2 
pencil. 

When recording your answer with a No. 2 pencil, fill jn the entire space between the 
dotted lines and the full length of the lines by pres: ing firmly on the pencil and 


using several strokes. 


IF YOU WISH TO CHANGE AN ANSWER, ERASE TIIE OLD ANSWER COMPLETELY. Do not cross it 
out. Then record your new answer in the proper space. 


Do not make any marks except your answer. More than one answer to a question or 
improperly erased answers or stray marks caused by’ resting the pencil on the paper 


may be counted as wrong answers. 


Use pen (blue or black ink) for filling in the headings on your answer sheet but use 
only the No. 2 pencil for recording your answers. 


Do not fold the answer sheet or damage the edges because the rating is done by 
machine. 


Since there are only 4 answers listed for each question, the spaces to the right of 
the letter E on the answer sheet are not used for this examination. 


To help you understand the procedure, the following sample item is given: 
SAMPLE 0: The sum of 5 and 3 is 
(A) 2 (B) 8 (Cc) 9 oy) BE 


The sum of 5 and 3 is 8, so that the acceptable answer is shown thus on your answer 
sheet: 


SAMPLE 0: A a=sse § anos C seeee D s=ean 


CON TDN UE OF 
FOLL WING 


PAGE 


Page 2 
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Of the following, the best time for a supervisor to give advice about a job-related 
problem which a subordinate has brought up during an interview is usually after the 


(A) subordinate has told him all the facts 

(B) supervisor has determined the employee's unconscious motives for 
bringing up the problem 

(C) employee has submitted a written report on the problem 

(D) supervisor has discussed the problem with his superior. 


Of the following, the situation in which a supervisor would have to make the grreatest 
effort in order to communicate effectively with his subordinates would occur when 


(A) there is a large gap between the supervisor's background and experience 
and that of his subordinates 

(B) the subordinates have already learned about the information through 
informal channels 

(C) the subordinates have completed their education much more recently 
than the supervisor 

(D) the supervisor has been with the organization for a much shorter 
time than most of his subordinates. 


f the following, the factor which would be most critical in influencing whether 
subordinates accept or resent a supervisor's authority is the 


(A) manner in which the supervisor uses his authority 

B) frequency with which the supervisor ignores minor violations of rules 
degree of delegation to subordinates by the supervisor 

(0) cultural attitudes of individual subordinates toward authority. 


In which one of the following situations would employees be most likely to accept 
temporarily difficult working conditions without excessive complaining? 


(A) The organization has a strict policy of disciplinary action against 
uncooperative employees. 

(B) Employees do not have the right to take part in "job actions" or strikes. 

(C) An atmosphere of mutual trust and good human relations exists between 
subordinates and managerial personnel. 

(D) Relationshins between subordinates and managerial personnel are strictly 
businesslike. 


Assume that an agency has been reorganized into integrated work teams. Instead of 
assigning employees performing the same task to a single unit, such as a typing 
pool, those performing different but interdependent parts of an activity are put 
into the same work group. Of the following, the most probable result of such a 
reorganization would be to 


(A) permit more efficient work scheduling (C) decrease training costs 
(B) achieve greater economy (D) improve employee job satisfaction. 


The need for identification with a work group has been found to be one of the most 
powerful on-the-job motivations. Of the following, the employee who is least likely 
to have a strong attachment to his work group is one who 


(A) is at the very bottom of the organization's promotional ladder 

(B) belongs to a small department 

(C) works with others of similar background and interests 

(D) has worked for the organization for a considerable period of time. 
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According to many management experts, the one of the following situations which 
would be the most significant indication that employees of an organization are 
dissatisfied with their supervisors and feel that they are being treated unfairlv 
is one in which 


\) employees submit a large number of work-related suggestions 

) many employees are unproductive and seem to be continually 
loafing on the job 

(C) union membership has recently increased 

(D) turnover is low in spite of a comparatively good labor market. 


supervisor who has informal, friendly relationships with his subordinates is 
conducting himself 


(A) appropriately; good informal relationships set the stage for 
better communication between the supervisor and subordinates 
on work-related problems 
(B) inappropriately; subordinates who have informal relationships with 
their supervisor are not likely to accept his authority 
(C) appropriately; friendly relationships between the supervisur and 
his subordinates will create a true feeling of equality between them 
(D) inappropriately; subordinates are likely to become suspicious of 
insincerity and fearful of being manipulated. 


Specialization is a commonly-used method of increasing productivity and efficiency 
in a large organization. Task specialization means that separate and comparatively 
simple parts of a more complex job are performed by different employees. Of tb 
following, this type of specialization probably would not 


(A) reduce training costs 

(B) permit the use of more specialized equipment 

(C) simplify the development of job controls 

(D) give most employees a greater sense of accomplishment. 


Some management experts who have studied informal communication patterns jn large 
organizations believe that the office grapevine is an effective means of communication. 
Of the following, an important function performed by the grapevine is to 


(A) permit feedback and spread information faster than most formal 
communication systems 

(B) give employees important information from reliable sources 

(C) permit management to identify rumor-mongers and troublemakers 

(D) bring informal leaders to the attention of management. 


Recent studies of morale and productivity tend to show that 


(A) the correlation between morale and productivity is rather low 
(B) high morale is associated with high productivity 

(C) low morale is associated with high productivity 

(D) low morale is associated with low productivity. 


Research studies have indicated that teamwork among employees is most likely to 
result in higher productivity in a situation where 


(A) employees accept as legitimate management's demands for higher productivity 
(3) management strongly encourages the workers' competitive spirit 

(C) employees are unified for the purpose of protecting themselves against 
management's demands 


(D) management does not encourage employees to make independent decisions. 
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The relationship between boredom on the job and fatigue is correctly stated as 
follows: 


(A) Boredom usually results in increased fatigue. 

(B) A worker usually becomes bored when he expends a minimum of 
physical energy. 

(C) Fatigue usually results in boredom. 

(D) A worker who is bored does not usually become fatigued. 


The "halo effect" can properly be suspected of harming supervisor-subordinate 
relationships when the supervisor 


(A} does not discriminate between the good and poor work of an emp loyee 
considered by him to be generally superior 

(B) expects his subordinates to treat him in an impersonal and formal 
manner 

(Ch hesitates to discipline employees because of an extreme need for 
them to like him 

(D) is unable to gain his employees' confidence because he cannet shed his 
reputation for being hardboiled and unfair. 


It is generally considered that the best interview is the one in which the 
interviewer talks less than the person interviewed. The one of the following 
which is an effective device to encourage the other person to talk during the 
interview is for the interviewer to 


(A) summarize the feelings the person has expressed, omitting details 
and incidentals 
(B) keep silent and show no indication of his reaction to what the person 
is saying 
(C) clearly show his approval or disapproval of what the person is saying 
(D) talk to the person in terms of concepts rather than specifics. 


Studies of groups of workers doing the same job under the same conditions have 
shown that there are always a few workers who have more accidents than the rest. 
The one of the following which is least likely to be a finding of such studies 
is that those who have the most accidents probably are 


(A) middle-aged (C) inexperienced at the job 
(B) poorly adjusted to work (D) less efficient than other workers. 


Work measurement has been defined as ''the determination of the proper amount of time 
and effort required for the effective performance of a specific task." Of the 
following, the factor which would be least relevant in studying an operation by 
means of work measurement is whether the operation is 


(A) repetitive with constant standards of quality 
(B) compensated for at a prevailing rate of pay 

(C) routine in nature and relatively easy to perform 
(D) performed in large volume. 


Of the following, "participative management" can be defined best as a method in 
which 


(A) subordinates have formed groups for the purpose of gaining participation 
in the decisions of management 

(B) management makes a practice of encouraging subordinates as a group to 
discuss and participate in decisions on a wide variety of work-related 
problems 

(C) managerial employees are given varied assignments on a rotating hasis 

(D) management gives all employees the opportunity to participate in major 
policy decisions. 
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"Internalized motivation’ has been describid 19 1 vethwd »t motivating employees by 
enabling them to derive satisfaction through Joirp the icb itself. This approach 
to motivating employees would require management to 


(A) assume that most employees like work and enjov doiiz a good job 

(3) encourage competition among employees for promotions and higher salaries 
(C) emphasize improved fringe benefits and conditions of work 

(D) consider employee needs to be more important than organizational needs. 


supervisor should find that he must issue an order his subordinates will 
probably resist, it is advisable for the supervisor to first 


(A) discuss the order with his subordinates and give them an opportunity 
to ask questions and make objections 

(B) issue the order wittout comment and discourage disc. sion and objections 
by his subordinates 

(C) inform his subordinates that he does not agree with the order he is going 
to give them, but must carry out the decisions of higher authority 

(0) inform his subordinates that he will take disciplinary action against 
those who resist carrying out the order he will present to then. 


hen a supervisor finds that his subordinates differ considerably in the amount of 
attention and guidance they require of him, it would be most advisable for the 
supervisor to 


(A) ] his supervisory practices according to individual needs 
(fh) give an equal amount of attention and guidance to each subordinate 
in order to be fair 


(C) give less responsibility to subordinates who seek reassurance 

(D) permit employees who prefer independence to work strictly on their own, 
Connecting lines on an organization chart represent lines of 

(A) management quality controls (C) authority and responsibility 
(3) work flow (D) fiscal accountability. 


"General supervision’ has been defined as a method in which the supervisor makes 
assignments in broad, general terms and gives considerable autonomy to subordinates, 
in accordance with their knowledge and ahilities. A supervisor who uses this 

method is least likely to 

(A) do different work from that of subordinates 

(B) concentrate on long-range problems 

(C) exert excessive pressure on subordinates 

(D) devote considerable effort to training subordinates. 


Of the following, the most important reason why a supervisor should be cautisus 
about giving subordinates advice about personal problems is that the 


nm 
ww 


subordinate may blame him if the advice turns out to be misleading 
supervisor should not discuss personal problems with subordinates on 
office time 

(C) subordinate may lose confidence in his ability to perform on the job 
(D) supervisor may not know enough to give helpful advice. 


Professional or technical consultants may be used most appropriately by a human 
services agency to 


(A) direct staff conferences centered around programs 

(B) give advice regarding the quality of service or the effectiveness of plans 
(C) supervise implementation of programs they have developed 

(D) direct the in-service training program. 
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Yf the following, the most important aim t is to assist a 
{isadvantaged community to 


(\) accomplish particular tasks within the service 

(BE) evaluate its achievements in the light of '-°-.tace oni 

(C) prepare itself to function as an organized. -ffective entit: 

(0) collect information about the network of socia:s services 

The goal of the current reorganization of the New York City Human Resources 
Administration is to provide services in the neighborhoods whenever feasible, in 
accordance with the needs of neighborhood residents. Of the following, a desirable 


result of this would he 


(\) variation in the conbination of services offered in each neighborhood 

(%) uniformity of services offered in each neighborhood 

(C) considerable variation in the amount of community particination in 
each neighborhood 

(D) decrease in the need for utilization of existine community facilities. 


Appropriate division of functions and responsibilities between the communities and 
central headquarters is vital to the successful administration of a decentralized 
hovuan services agency in a large urban center. The one of the following which is 
not an appropriate function to be performed at the community level is 


) review of district plans for policy compliance 

(B) stimulation of maximum use of both new and e..isting resources 
(*©) review of service program gvals aad relationships 

(D) operation of community service teams. 


new approach to the delivery of services currently being developed by the 
New York City Department of Social Services is the Community Social Services Program. 
The one of the following which is not a major feature of this program is 


(A) stationing of personnel in locations that are easily accessible to clients 
(B) decentralization of authority to meke decisions regarding services 
(C) use of purchased services from private organizations to completent 
directly-delivered services ‘ 
(0) preparation of a single annual Human Services Plan applicable to all 
Human Resources Districts. 


The Community Social Services Program of the Department of Social Services is being 
implemented initially in the following !!uman Resources Districts: 


(A) The Lower East Side, Bushwick and Parkchester 

(B) East Yew York, Washington Heights, and the Lower East Side 

(C) Washineton Heights, the Lower Fast Side, and fedford-Stuyvesant 
(D) South Jamaica, Central Harlem and Williamsburr. 


Community participation is a Key feature of the Community Social Services Program of 
the Department of Social Services. Of the following, a correct statement with regard 
to the community participation facet of the program is that it 


(A) extends to the District level the consumer involvement aspect which was 
previously embodied in the client advisory committees of the City's social 
service centers 

(B) implements federal and state regulations which call for community control of 

planning and evaluation of services 

emph3sizes participation of local residents in service delivery rather than 

i. planning and evaluation 

( axes evaluation the responsibility of consumers of social services. 
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the citizens of a disadvantaged community generally have a positive belief that 
can work together in solving their problems, and thus achieve worthwhile goals, 
are usually said to be working toward a community that is 


complex (B) diverse (C) functional (D) underorganized. 


Which of the following is most likely to produce an initial positive response by 
local residents toward a community project? 


(A) 
(B) 
(Cc) 
(D) 


Adoption of the cultural mannerisms and typical dress of local residents, 
Endorsement by a person or an institution trusted by local residents. 
Solution of a problem for the community to show good faith. 

Promising solutions for all the problems facing the community. 


Of the following, the most common mistake made by participants in social 
controversy is to 


(A) 
(B) 
(Cc) 
(D) 


attribute a single point of view of all people of a certain group 
try to understand all sides of the controversy 

play down the controversy, in the hope that it will disappear 
expect a cooperative resolution of the controversy. 


Of the following, the most probable result of citizen involvement in a community 
crisis situation is 


(A) 
(B) 
(Cc) 
(D) 


stronger identification with community organizations and institutions 
decreased effectiveness {n the operation of community organizations 
more accurate reporting of community events by mass media 

loss of confidence in public agencies and private social services. 


Research in the form of fact finding constitutes a necessary foundation for 
community planning and action. The one of the following which is least likely to be 
a purpose of this type of research is to 


(A) 
(B) 
(Cc) 
(D) 


evaluate services 
determine needs 
establish priorities 
develop a theory. 


"Administrative decentralization' gives more authority to City agency chiefs at the 
‘district level, as contrasted with ‘political decentralization,’ which gives more 
decision-making power to local residents. Of the following, the most important 
reason for emphasizing administrative decentralization would be to 


(A) 
(B) 
(C) 


(D) 


remove decision-making power from local groups, because recent crises 
have shown that neighborhoods are not yet ready for community control 
provide assurance that district chiefs of City agencies coordinate 
their actions with their central offices 

encourage community participation in those neighborhoods where local 
residents have not taken the initiative to become involved in decion- 
making 

provide essential administrative control at the neighborhood level in 
order to give community groups more assurance that their projects can 
be carried out. 
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The “outreach function provides a means of reaching out to disadvantaged 

community residents in order to become familiar with their needs and problems and 

to inform them of services available. Which of the following would probably perform 
this function most effectively in their own neighborhoods? 


(A) community leaders (C) small businessmen 
(B) paraprofessionals (D) school teachers. 


When people of a disadvantaged community are’ given the opportunity to gain satis- 
faction and self-confidence from small accomplishments, the most probable result is 


(A) discouragement of vrogress toward major goals 

(B) development of a false sense of achievement 

(C) progress toward overcoming more difficult problems 

(D) development of unrealistic and unachievable ambitions. 


At times it is necessary for a community group to fourm a tempoxary work alliance 


with other self-interest community groups for the mutual support df programs. Such 
a temporary work alliance is usually called a 


(A) federation (C) union 
(B) coalition (D) society. 
YON TO ‘T PAGE 
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Answer questions 41 through 45 solely on the basis of the following paragraph. 


With the generation gap yawning before us, it is well to remember that 20 years ago 
teenagers produced a larger proportion of nonwedlock births than today, and that 
the illegitimacy rate among teenagers is lower than among women in their twenties 
and thirties. In addition,the illegitimacy rate has risen less among teenagers 
than among older women. 

It is helpful to note the difference between illegitimacy rate and illegitimacy 
ratio. The ratio is the number of illegitimate babies per 1,000 live births. The 
rate is the number of illegitimate births per 1,000 unmarried women of childbearing 
age. The ratio talks about babies, the rate talks about mothers. The ratio is 
useful for planning services, but worse than useless for considering trends, since 
and the fertility of married women. For example, the ratio among girls under 18 is 
bound to be high in comparison with older women, since few are married mothers, 
Vowever, the illegitimacy rate is relatively low. 


Of the following, the most suitable title for the foregoing pa.c«ge would be 


(A) The Generation Gap 

(B) Moral Standards and Teenage Illegitimacy Ratio 

(C) A Comparison of Illegitimacy Rate and Illegitimacy Ratio 
(D) Causes of High Illegitimacy Rates. 


/-cordine to the passage, which of the following statements is correct? 


(A) The illegitimacy rate has fallen among women in their thirties. 
(B) The illegitimacy ratio is the number of illegitimate births 
per 1,000 unmarried women of childbearing age. 
(C) The illegitimacy ratio is partially dependent on the illegitimacy rate. 
(D) The illegitimacy rate is more useful than the ratio for planning services. 


According to the passage, of o. swing age groups, the illegitimacy ratio would 
be expected to be highest in -omparison with the other groups for the group aged 


(A) 17 (B) 21 «c) 25 (@) 29. 


According to the passage, of the following age groups, the illegitimacy rate would 
be expected to be lowest in comparison with the other groups for the group aged 


(A) 17 (B) 21 (Cc) 25 (D) 29. 


As used in the foregoing passage, the underlined word composition means, most nearly, 


(A) essay (B) makeup (C) security (D) happiness, 
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Answer questionga 46 throurh 29 solely on the basis of the following paragraph. 


In countins the poor, the “ocial Security Administration has developed two poverty) 


thresholds that designate families as either poor’ or ‘near poor. The Administra 
tion assumed that the poor would spend the same pronrortion of income on food as t 
rest of the population but that, obviously. since their income was smaller, thei 
range of selection would be narrower. Im the Low Cost Food Plan t mount allocate 
to food from the average expenditure was cut to the minimum that the Agriculture 
Pepartment said could still provide American families with an adequate diec. Thi 
Low Cost Plan was used to cliaracterize the ‘near noor caterorv, and an even lk 
Leonomy ‘ood "lan was used to characterize the poor caterorv. The Leonony Foo 
Plan was ased on 79 cents a person for food each day, assuming that all food we 

be prepared at home. The Agriculture Department estimates tiiat on!y alout lu per 
cent of rersons spending 70 cent or less for food each da ictuall ere able t 


ret a nutritionally adequate diet. 


’ the followin the most suitable title for the foregoine paragray woul 
(A) The uperiority of the tconom ‘lan over the Lost Cost lan 

( The ‘eed for a ‘utritionally Adequate Diet 

(C) Food xpenditures of the Poor and the Neer Poor 

(9) iet n the United “State 


\ccording to the paragraph, the Social Securitv Administration assumed, in se 
its noverty levels, tiat the 


spenl a smaller proportion of income for food than thie averare non OT 
(8) would not eat in restaurants 


(C) i y vrounp inclu only those wit’) a nutritionally inadequate iet 


e: send more monev on food than the near poor. 


\ccording to the pararrapl, it would be correct to state tiat the Low Cost F 
Plan 


(A) is ahove the minimum set Ly the Avriculture bDenartment for a nutritionally 


adequate diet 
(.) gives most people a nutritionally inadequate diet 
(C) is lower than the fconomy Food Plan 


hy 


(") represents the amount spent the "near poor. 


s estimated by the Department of Acriculture f 


, the nercentage of people snendine 
7) cents or less a day for food who did not get a nutritionally adequate diet wa 
(A) 190 (BR) 90 (Cc) lv (D) 9? 
As used in the foregoinre narapraph, ‘the u rlined vords allocated to mean, most 
nearly, 
(A) offered for (i) assigned to (C) wasted on (D) spent on. 
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Answer questions 51 through 55 solely on the basis of the graphs presented below. 
Note that the unemployment rat for employables who move into Central City is the 
same as for the City as a whole r 


Emplovables z 5 a 
in Thousands Employahles ho Moved into Central City, 1960-1971 
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51, The ratio of unemployed to employed among the employables who moved to Central 
City in 1961 was 


(A) 1 to 3 (B) 1 te 12 (C) 1 te 24 (D) 1 to 36. 


52. In 1972, if the number of employables entering the city increases to 33,000 and the 
unemployment rate is half the rate of 1971, the number of unemnloyed among the 
employables who move to Central City in 1972 will be 


(A) 825 (B) 1,650 (C) 8,250 (D) 16,500. 


53. Of the following, the year with the greatest number of unemployed among employables 
who moved to Central City was 


(A) 1964 (B) 1965 (C) 1967 (D) 1971. 


54. Of the following,the period during which the greatest change occurred in the 
number of unemployed among emplovables who moved to Central City was 


(A) 1961 and 1962 (B) 1963 and 1964 (C) 1969 and 1970 (%) 1970 and 1971. 


55, If there are 640 unemployed among the 23,470 people who move to Central City in 1975, 
the Central City unemployment rate in 1975 will be most nearly 


(A) 2% (B) 3% (C) 42 (D) 5%. 
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The Chairman of the City's Commission on Human Rights has stated that the purpose 
the ethnic survey of City employees conducted recently wes to determine 


(A) the degree of "ethnic group identity" among City employees belonging 
to various minority groups 

(8) whether there has been job discrimination by City agencies against 
minority groups 

(C) the basis for the establishment of quotas which would guarantee jobs 
to a certain number of minority group members 

(D) a method of strengthening the merit system through establishment of 
discretionary hiring practices for minority group members. 


Of the following, an important purpose »f the separation of the functions of income 
maintenance and social services in the Department of Social Services is to 


(A) give caseworkers more time to provide services and counselling 
to welfare families and individuals 
(B) eliminate investigation of all families and individuals applying 
for public assistance 
(Cc) enable the Department of Social Services to implement the State- 
mandated program of check pickup at the lew York State Employment Service 
(D) facilitate decentralization of the allocation of funds for services. 


Changes in the Food Stamp Program made by new regulations passed iis spring 
resulted in 


(A) raising both the maximum income for non-public assistance families who 
would be eligible to purchase stamps and the bonus for families already 
participating, but also raising the cost of the stamps 

B) lowering both the maximum income for non-public assistance f:..ilies who 
would be eligible to purchase stamps and the bonus for families already 
participating, but lowering the cost of the stamps 

(C) raising the maximum income for n -public assistance families who would 
be eligible to purchase stamps and lowering the cost of the stamps, but 
lowering the bonus for families already participating 

(9) lowering the maximum income for non-public assistance families who would 
be eligible to pur’ ase stamps and raising the cost of the stamps, but 
also raising the bonus for families already participating. 


The Federal Revenue Act of 1971 includes special benefits for working mothers who 
have child-care expenses in that it provides for deduction of 


(A) specific household and child-care expenses for single, divorced or 
widowed parents only 
(B) specific household and child-care expenses for dependents under 15 
(C) carfare and other expenses relevant to the mother's employment 
(D) allowance for an additional dependent for the person ciring for the child. 


A serious objection raised by those who consider the proposal for revenue sharing 
by the Federal government to be a major threat to welfare reform is that 


(A) states and localities would not have sufficient conrrol over how the 
money is spent 

(B) increased state and local control over expenditures has consistently 
worked against disadvantaged and minority groups 

(C) there are no Federal provisions for enforcement of compliance wich 
regard to discrimination for reason of race, color or national origin 

(D) the state governors would not be able to set their own priorities or 
monitor violations by local officials. 
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According to the Department of Social Services, of the following, the smallest 


number of recipients of public assistance consists of > 
(A) children under 21 (C) employables 
(B) aged adults (D) adults caring for others. 


Responsibility for quality control studies of public assistance clients' eligibility 
in New York City by means of interviews and analysis of case records is being taken 
over by the 


(A) U.S. Department of Health, Education and Welfare 

(B) New York State Department of Social Services 

(C) New York City Department of Investigation 

(D) Bureau of Income “Maintenance of the Human Resources Administration. 


The one of the following which is not a current method being used by the Human 
Resources Administration in its efforts to reduce welfare costs is 


(A) improvement of managerial capability 

(B) simplification of procedures 

(C) insistence on greater accountability 

(D) full investigation of all applicants for assistance. 


The one of the following which would be most likely to reduce the tendency of the 
unemployed poor to migrate to urban centers is 


(A) increased federal subsidies to the small farmer 
(B) unionization of migrant farm workers 

(C) a national guaranteed income mai:utenance program 
(D) an increase in the national minimum wage. 


According to the most recent U.S. Census Bureau Report, economic and educational 
gains made between 1960 and 1970 by Puerto Ricans living in New York City, as 
compared with the Blacks, were 


(A) considerably greater (C) about the same 
(B) considerably smaller (D) slightly greater. 


Of the following modern child-care experts would consider the least desirable 
setting for treatment of most juvenile offenders while under professional supervi- 
sion to be 


(A) the family's home (C) a foster home 
(B) a greup residence (D) a large institution. 


New York State has instituted a program of ‘work relief,'’ which requires all Home 
Relief recipients who are eligible to work, but cannot find jobs, to ‘work off" 
their welfare checks on non-salaried work projects. This approach has met with 
considerable opposition, mainly because those who oppose it believe that the 
recipients 


(A) may take away jobs which were formerly performed by paid employees 
(B) are unskilled and therefore qualified for an extremely limited number 
of work assignments 
(C) are likely to be resentful and inefficient in carrying out their assignments 
(D) will receive no improvement in financial status and no assurance of stable 
employment,so that they can be removed from the welfare rolls 
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A significant aspect of an amendment to the Social Security Act which became 
effective July 1, 1972 is that it introduces into the Act a program of 


(A) public service jobs (C) medical assistance to the aged 


(B) income maintenance (D) aid to aependent children. 


During the past few years, the U.S. Supreme Court has made a series of key rulings 
that directly affect welfare clients. None of these rulings has been concerned 
with 


(A) fair hearings (C) home inspections 


(B) state residence requirements (D) the family assistance plan. 


A demonstration program for public assistance recipients in New York City recently 
approved by the U.S. Department of Health, Education, and Welfare which has been 
the subject of considerable controversy is the 


(A) Incentives for Independence Program (C) Work Incentive Program 
(B) Haitian Training Project (D) Demonstration Work Project. 


GO ON TO THE NEXT PAGE 
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, 
71. Acceptance of which of the following services provided by the Department of Social 
Services for public assistance clients is no. voluntary on the part of the clients? 


(A) Home management (C) Child protection 
(B) Family planning (D) Educational counselling. 


72. The one of the following which is not part of the organization of the Youth Services 
Agency is the 


(A) Neighborhood Youth Corps (C) Youth Development Centers 
(B) Recreation Support Program (D) Youth Vocational Training Program. 


73. The one of the following which is not included under the recently organized Office 
of Special Services for Adults in the Department of Social Services is 


(A) senior citizen programs (C) foster care for the aged and disable 
(B) Medicare (D) adult institutions. 


74. Several juvenile detention centers formerly run by the New York State Office of 
Probation are now operated by the 


(A) Youth Services Agency (C) Office of Child Development 
(B) Department of Social Services (D) Judicial Conference. 


75. Decentralized administration of the City's anti-poverty efforts is provided by the 


(A) Community Corporations (C) Human Resources District Councils 
(B) Council Against Poverty (D) Neighborhood Youth Corps. 
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The component of the Human Resources Administration which is authorized by the 
Mayor as the agency to receive community action funds from the federal government 
is the 


(A) Council Against Poverty (C) Manpower and Career Development Agency 
(B) Community Development Agency (D) Agency for Child Development. 


The Equal Employment Opportuni.y Commission is correctly described as the 


(A) Federal agency which acts on charges of discrimination in employment 

(B) State agency which operates employment and specialized placement offices 
(C) City agency which develops job opportunities for underemployed persons 
(D) City agency which acts on charges of discrimination in employment. 


The recent reorganization of the Youth Services Agency emphasized administrative 
changes that would allow for 


(A) expansion of counselling ,psychological and psychiatric services to 
anti-social youth 


(B) greater community responsibility for and participation in the 
delivery of services to youth 


(C) more emphasis on direct services to street youth and fighting gangs 
(D) tablishment of youth narcotics addiction prevention and treatment programs. 


The basic reason for the establishment of the Agency for Child Development as part 
of the Human Resources Administration was to 


(A) provide a single agency to consolidate and administer programs for 
preschool-age children 


(B) establish a Commission to insure maximum parent and community involvement 
in programs for children 


(C) take over the licensing of hoth public and private programs for preschool-age 
children 


(D) insure a diversity of Programs to meet the needs of a broad spectrum of 
children, 


The main functions of the Manpower and Career Development Agency (MCDA) are to 


(A) run manpower and recruitment centers under contract with private organizations 

(B) train the unskilled, upgrade existing skills, develop job opportunities, and 
and place newly-trained people in jobs 

(C) provide remedial education and follow-up for disadvantaged potential college 
students, vocational counselling and testing for veterans and ex-addicts 


(D) provide job development, interviewing and placement, and manpower research 
services. 


P/SUP 
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ANSWER SHEET COLLECTION: You may take the question booklet with you. When you 
have completed the test, remain seated and summon the monitor. The monitor will 
pick up your answer sheet. Leave the building quickly and quietly. You should 
make a note of your application number and include this number in any future 
correspondence with the Department of Personnel with respect to this examination. 


KEY ANSWERS AND PROTESTS: The proposed key answers to this test will be posted 
on the Official Notices Bulletin Board of the Application Section of the 
Department of Personnel, 49 Thomas Street, New York, N.Y., beginning 10/25/72 
for a period of three days. Candidates who wish to file protests against the 
Proposed key answers must submit in writing such protests together with their 
reasons for such protests. Such protests must be postmarked not later than 
11/15/72, and must contain the candidate's application number. Protests 
should be addressed to the Department of Personnel, 55 Thomas Street, New York 
N.Y. 10013, with the examination title written on the face of the envelope. 
Protests postmarked later than 11/15/72 will not be considered. 


Protests should be clear and legible, preferably typewritten. Each protest must 
begin on a separate page and state the number of the question being: protested. 
Protests should begin by indicating which answer the candidate considers correct, 
and should present reasoned arguments and clearly identify any quoted sources 

Or author’ties. Each page must be signed by the candidate and state his address. 
Only protests submitted by individual candidates will be considered. 


After the Department of Personnel h.s considered all protests, properly filed, it 
will make a final determination thereon and will thereafter post the final key on 
the Official Notices Bulletin Board of the Application Section of the Department 
of Personnel for a period of three days. Such posting constitutes official public 
notice by the Department of Personnel to all candidates of the final determination 
in this connection. 


The Department of Personnel, in establishing key answers to this test, reserves 
the right to determine which of the answers listed for each question is to be 
deemed and accredited as acceptable, and whether more than one of the answers 
listed for each question is to be deemed acceptable and accredited as such. The 
Department further reserves the right to cancel and annul any question whenever, 
upon inquiry, it deems that none of the listed answers thereto can properly be 
considered acceptable. 


Please do not telephone the Department of Personnel requesting information as to the 
Progress of the rating of this examination. Such information will not be available 
to candidates. All candidates in this examination will be notified individually 

by mail of the rating they receive when the rating of all candidates in the 
examination has been completed. Notify this department promptly in writing of any 
change in your address to insure prompt delivery of such notice. 
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z . 7 sail OFFICIAL 
[Seconp AMENDED Notice oF EXAMINATION NuMBER 2013] Cal.0678-9.28.72 
Experience Form A Notice of Vxamtauation 56040 
: 2nd Amended Notice ; 
(September 29, 1972) 
No, 2013 
SUPERVISING HUMAN RESOURCES SPECIALIST 
Cai dates who filed previously need not file again, 
“y J “ores 
In conjunction with the holding of this examination, departmental promotion examinations will 
be held, Although names appearing on the promotion lists will receive prior con- 
sideration, it is expected that there will be a sufficient number of vacancies so 


that the open-comperitive list will be used. 


Candidates who wish to take a portion of the written test in a particular forecisn 


language’ must indicate their choice of lanpuare on the anplication form. If at 
least five percent of the applicants for the written test request that it be ¢ 
in a specific foreign lanpuape, 25% of the questions will be printed in the specific 


foreign language, in addition to being printed in English. 


SALARY AND VACANCLES: The appointment rate for this position is $12,500 per annum. 
Numerous vacancies exist in the Human Resources Administration, its constituent 
agencies and otner City departments. 


APPLICATIONS: Received on September 29 and October 2, 1972, in person 

only, in Room 219, Department of Personne], 55 Thomas Street, New York, 

N.Y. 10013, from 9:30 a.m.to 4:.m. Ho applica ions issued er received by mail. 
a 


DATE OF TEST: The written test is expected to be held on October 14, 1972. This 
date is tentative only and may be changed if circumstances so demand. 


PROMOTION OPPORTUNITIES: Employecs in the title of Supervisi,« Human “esources 
Specialist are acco’ ded prunotional opportunities, when eligible, to ' title of 
Principal Human Resources Specialist. 


MINIMUM RECUIREMENIS: A baccalaureate degree issued after completion of 

a four-year course in an accredited collere, and three years of full-time 

experlence ir. a professional, technical or administrative capacity in one or 

more of the tullowing areas; community planning, development, organization or rel 

educational counseling or testing; intergroup relations; business, personnel or E:ilic 

administration; fiscal or financial control: journalism; industrial, labor or pullic 

relations; uri. .n development or renewal; or sccial, health or educational action or 

planning, or a satisfactory equivalent combination of education and experience. lovwever, 
at least one year's experience must have been in the direct servicing of a disadvantaged 

client population similar to that serviced by the Human Resources Administration or 

one of its constituent agencies in a Program relating to community action or development, 

manpower and skills development, or educational or social change, or in the direct manape- 

ment or technical support of such a program. All candidates must meet the specialized 
one year's experience requirement as cited above. Relevant professional, technical or 

administrative experience may be substituted for college or high school education on a 

year for year basis. Graduate study in an accredited college may be substituted for up to 

one year of the required general experience. 


Experience Form A must be filed with the application. Candidates are advised to m. 

full and complete Statements caencerning their volunteer, part-time and full-time 
experience on the Experience Form A... If additional space i8 required, candidates can use 
additional blank pares. 


The minimum requirements must be met by the last date for the receipt of applications, 
All candidates who file applications will be summoned for the written test prior to a 
determination of whether or not they mect the minimum requirements. The experience 
Papers of passing candidates only will be evaluated. 

DUTIES AND RESPONSIDILITIES: Under direction, with broad discretion for the use of in- 
dependent judgment and initiative, supervises and directs 4 group or groups of Senior 
Human Resources Specialists and lower-level employees of the Hluman Resvurces Occupational 
Group and/or other employees, in planning, executing and evaluating programs relative 
to one or more of the following areas, manpower and career development, management, 
community development, or program planning and analysis; or performs complex and highly 
responsible staff assignments in relation to any of the programs listed above; performs 
related work. 
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Supervises a proup or groups of Senior luman Resources 
] level employees of the Human Resources; Occupational Group and/or 
nninege, executing aad evaluating any one or several of the above 
mssigns and reviews the regular and special work of the group 


1 + and instructs members of the proup; coordinates ali the 
t | reports, including research, of assipned yroup; develops and 
used, policy procedures and overall plan of operations; as- 


and priorities; adopts and translates research knowlcdre 
maintains liaison with business, labor and industrial 
» private and community orpaniz 


© on program policies, plans and budgets; seeks improvement in 
' ! ; reviews compliance with terms of contratts; may provide pr« 
area of specialty; interviews and participates in tie 
l employces, clier 
' istant director in formulating policy; prepares sunmarie 


rofessional or clerica 


ts and trainces; 


itions and arencies to nepotiate 


evaluation reports for use by the director and/or commissi 


€ 


te or civic organizations; designs, develops and implements 
tive and far-reachi problems and policies related t 
ts with officials and respresentatives of private and publi 
roups to accomplish any of the above, to provide functional 
feedback as to policies, proprams and procedures; may assume th 
liate supervisor in the absence of that individual. 
U0, 70% required; oral, weir':t 50, 70% required. The written 
t ltiple-choice type and inc e questions in the following 
1 upervising Human ?e é falist; ictions of relevant 
t current developments ar lems in the field of human 
it 1 tio ind organization; jud t in on-the-job situations; 
ff dl lopment, supervision and eui *; language usape, includin 
prehension; arithmetic cor tions and interpretation of 
and other related areas. 
he oO. st will be speech, inner vl judgment. fhe oral te 
iT lity to respond to stres ituations, to handle and 
ef fe ly with high adninistrative officials, to conduct confercnces 


init oups, etc. Only those candidates who 
for ! Oral test, 


pass the written test 


secti f the General Examination Rerulstions and the General Provisi 
of 1ation are also part of is Notice 

I WMILSSLON: Harry I. Bronstein, Chairman; 
Jr., Secretary, 
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[Examination Number 2013) Application N 
cation No. 


Poom “In. 
Seat Yo. 
City of New York 
Department of Personnel 


EXAMINATION NO. 2013 - OPEN COMPETITIVE (0.C.) EXAMINATION FOR 
SUPERVISING HUMAN RESOURCFS SPECIALIST 


EXAMINATION NO. 2012 - OPEN COMPETITIVE (0.C.) EXAMINATION FOR 
SUPERVISING IIUMAN RESOURCES SPECIALIST (MANPOWER DEVELOPMENT AND TRAINING) 
Date: October 14, 1972 Total Time Allowed: 2% Hours 
"ritten Test - Weight 59, 70% Required 
GENERAL INSTRUCTIONS 


Fill in your application .umber, room number, and seat number at the top of this page. 


BELLS: First Bell: Test booklet given out. READ THIS PAGE ONLY. 
Second Bell: Turn the page and begin work. 
Third Bell: End of test. Stop all work. This bell is run, 2's hours 
after the second bell. If.you finish before, raise your 


hand to let the monitor know. 


TEST BOOKLET: This test booklet has both tests and there should be 21 pages. 
All questions are of equal weight heck your test booklet 
after the second bell to be sure it has all the pages and is 
not defcctive. 


ANSWER THE QUESTIONS AS INDICATED BELOW: 


IF YOU ARE TAK1Nu ONLY EXAM. NO. 2013 - 0.C. SUPERVISING HUMAN RESOURCES 
SPECIALIST: ANSWER QUESTIONS 1-80 
IF YOU ARE TAKING ONLY EXAM. NO. 2012 - 0.C. HUMAN RESOURCES SPECIALIST (‘DT): 
ANSWER QUESTIONS 1-70 and 81-90 


OR: If you are taking two or more tests follow the special instructions 
you have been given concerning estions to answer and time allowed. 


You must hand in your test booklet, answer sheet, and all scrap 
paper after you finish. You may take nothing from the room 
with you except the "Candidate's Personal Record of Answers” 
sheet, on which you may make a record of your answers for future 
reference. 


WARNING: Smoking is NOT allowed. You will be disqualified if you copy 
: answers from anyone or use books or notes. 


It is against the law to take the test fer somebody else or to 
let somebody else take the test for you. 


This test booklet is the property of the Department of Personnel and 
you must turn it in at the end of the test. It is worth mre 
than $250. You will be subject to penalty if you do not turn it in. 


After the test star \ obody can leave till after 10:30 a.m. 
Nobody can come in after 10:30 a.m. If you want to drop out of 
the test and do not want to have your answers rated, write 

'T withdraw’ on your answer sheet and sign your name. 


DO NOT TURN THE PAGE UNTIL THE SECOND BELL IS RUNG 
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DIRECTIONS FOR ANSWERING QUESTIONS 


Answer all questions on the official Answer Sheet before the third bell has rune. 
"OUR ANSWER SHEET WILL BE RATED. 


tr each question, select that answer which you believe to be the acceptable one 
yn se listed and note the capital letter preceding your selected answer. 

official answer sheet, find 
id record your answer by 


the row numbered the same as the 
the right of the 


space between the dotted 
representing your 


blackening the 
capital letter 


questior 
lines immediately 
selected answer, using a No. 2 


en recording vour answer with a pencil, fill in the entire space between the 
Jotted lines and the full length of the lines by pressing firmly on the pencil and 
using several strokes. 
I} Mt! WTSH TO CHANGE AN ANSWER, TRASE TIE OLD ANSWER COMPLETELY. Do not cross it 
ut Then recotd your new isver in the prover space. 
rt ame any arks except your answer. lore in one answer to a question or 

roper erased answers or stray marks cause 

1av 


nly the No. 2 pencil for recording 


Y resting the pencil on the paper 
r ounted as wrong answers. 


pen (blue or black ink) for filling in the headings on your answer sheet but 


use 
your answers. 
t fold the answer eet or damage the edges because the rating is done by 
1e. 
e there are only 4 answers listed ech question, the spaces to the right of 
Jetter f on the answer sheet are not used for this examination. 
ielp you understand the procedure, the following sample item is given: 
ANPLE 9: The sum of 5 and 3 is 
(A) 2 (B) 8 (Cc) 9 } ‘1d. 
sum of and 3 is %, so that the acceptable answer is shown thus on your answer 
H¢:3 
NMIPLE ° A s2=e= Sera Cc 
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Of the following, the beit time for a supervisor to give advice about a job-related 
problem which a sv ordinate has brought up during an interview is usually after the 


(A) subordinate has told him all the facts 
-(B) supervisor has determined the employee's unconscious motives for 
bringing up the problem 
(C) employee has submitted a written report on the problem 
(D) supervisor has discussed the problem with his superior. 


Of the following, the situation in which a supervisor would have to make the greatest 
effort in order to communicate effectively with his subordinates would occur when 


(A) there is a large gar between the supervisor's background and experience 
and that of his subcrcdinates 

(B) the subordinates have already learned about the information through 
informal channels 

(C) the subordinates have completed their education much more recently 
than the supervisor 

(D) the supervisor has been with the organization for a much shorter 
time than most of his subordinates. 


Of the following, the factor which would be most critical in influencing whether 
subordinates accept or resent a supervisor's authority is the 


(A) manner in which the supervisor uses his authority 

(B) frequency with which the supervisor ignores minor violations of rules 
(C) degree of delegation to subordinates by the supervisor 

(D) cultural attitudes of individual subordinates toward authority. 


In which one of the following situations would employees be most likely to accept 
temporarily difficult working conditions without excessive complaining? 


(A) The organization has a strict pulicy of disciplinary act_on against 
uncooperative employees. 

(B) Employees do not have the right to take part in "job actions" or strikes. 

(C) An atmosphere of mutual trust and good human relations exists between 
subordinates and managerial personnel. : 

(D) Relationshins between subordinates and managerial personnel are strictly 
businesslike. 


Assume that an agency has heen reorganized into integrated work teams. Instead of 
assigning employees performing the same task to a single unit, such as a typing 
pool, those performing differeat but interdependent parts of an activity are put 
into the same work group. Of the following, the most probable result of such a 
reorganization » .uld be to 


(A) permit more efficient work scheduling (C) decrease training costs 
(B) achieve greater economy (D) improve employee job satisfaction. 


The need for identification with a work group has been found to be one of the most 
powerful on-the-job motivations. Of the following, the employee who is least likely 
to have a strong attachment to his work group is one who 


(A) is at the very bottom of the erganization's promotional ladder 

(B) belongs to a small department 

(C) works with others of similar backeriund and interests 

(D) has worked for the orpanization for a considerable period of time. 
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According to many management experts, the one of the following situations which 
would be the most significant indication that employees of an organization are 
dissatisfied with their supervisors and feel that they are being treated unfairly 
is one in which 


(A) employees submit a large number of work-related suggestions 
(2) many employees are unproductive and seem to be continually 
loafing on the job 

(C) union membership has recently increased 

(D) turnover is low in spite of a comparatively good labor market. 


A supervisor who has informal, friendly relationships with his subordinates is 
conducting himself 


(A) appropriately; good informal relationships set the stage for 
better communication between the supervisor and subordinates 
on work-related problems 
(B) inappropriately; subordinates who have informal relationships with 
their supervisor are not likely to accept his authority 
(C) appropriately; friendly relationships between the supervisor and 
his subordinates will create a true feeling of »quality between them 
(D) inapprovriately; subordinates are likely to become suspicious of 
insincerity and fearful of being manipulated. 


Specialization is a commonly-used method of increasing productivity and efficiency 
in a large organization. ‘Task specialization means “bat separate and comparatively 
simple parts of a more complex job are performed by different employees. Of the 
following, this type cf specialization probably would not 


(A) reduce training costs 

(Bb) permit the use of more specialized equipment 

(C) simplify the development of job controle 

(D) give most employees a greater sense of accomplishment. 


Some management experts who have studied informai communication patterns jn large 


organizations believe that the office grapevine is an effective means of communication, 


Of the following, an important function performed by the grapevine is to 


(A) permit feedback and spread information faster than most formal] 
cormunication systems 

(B) give employees important information from reliable sources 

(C) permit management to identify rumor-nongers and troublemakers 

(D) bring informal leaders to the attentfun of management. 


Recent studies of morale and productivity tend to show that 


(A) the correlation between morale and productivity is rather low 
(B) high morale is associated with high productivity 

(C) low morale is associated with high productivity 

(D) low morale is associated with low productivity. 


Research studies have indicated that teamwork among employees is most likely to 
result in higher productivity in a situation where 


(A) employees accept as legitimate management's demands for higher productivity 

(3) management strongly encourages the workers' competitive spirit 

(C) employees are unified for the purpose of protecting themselves against 
management's demands 

(D) management does not encourage employees to make independent decisions. 
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The relationship between boredom on the job and fatigue is correctly stated as 
follows: 


(A) Boredom usually results in increased fatigue. 

(B) A worker usually becomes bored when he expends a minimum of 
physical energy. 

(C) Fatigue usually results in boredom. 

(D) A worker who is bored does not usually become fatigued. 


The "halo effect" can proper!» be suspected of harming supervisor-subordinate 
relationships when the supervisor 


(A) does not discriminate between the goo? and poor work of an employee 
considered by him to be generally superior 

(B) expects his subordinates to treat him in an impersonal and formal 
manner 

(C) hesitates to discipline employees because of an extreme need for 
them to like him 

(D) is wiable to gain his employees' confidence because he cannot shed his 
reputation for being hardboiled and unfair. 


It is generally considered that the best interview is the one in which the 
interviewer talks less than the person interviewed. The one of the following 
which is an effective device to encourage the other person to talk during the 
interview is for the interviewer to 


(A) summarize the feelings the person has expressed, omitting details 
and incidentals 
(B) keep silent and show no indication of his rection to what the ;erson 
is saying 
(C) clearly show his approval or disapproval of what the person is saying 
(D) talk to the person in terms of concepts rather than specifics. 


Studies ec groups of workers doing the same job under the same conditions have 
shown that there are always a few workers who have more accidents than the rest. 
The one of the following which is least likely to be a finding of such studies 
is that those who have the most accidents probably are 


(A) middle-aged (C) inexperienced at the job 
(B) poorly adjusted to work (D) less efficient than other workers. 


Work measurement has been define: as "the determination of the proper amount of 

and effort recnired for the effective performance of a specific task." Of the 

following, ‘ie factor which would be least relevant in studying an operation by 
means of work measurement is whether the operation is 


(A) repetitive with constant standards of quality 
(B) compensated for at a prevailing rate of pay 

(C) routine in nature and relatively easy to perform 
(D) performed in large volume. 


Of the following, "participative management"' can be defined best as a method in 
which 


(A) subordinates have formed groups for the purpose of gaining participation 
in the decisions of management 

(B) management makes a practice of encouraging subordinates as a group to 
discuss and participate in decisions on a wide variety of work-related 
problems 

(C) managerial employees are given varied assignments on a rotating basis 

(D) management gives all employees the opportunity to participate in major 
policy decisions. 


time 
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"Internalized motivation" has been deser{h ss. °« 4 mei 4 of motivating employees by 


enabling them to derive satisfaction thre. teri rh ob itself. This approach 
to motivating employees would require management te 


(A) assume that most employees like work and entoy; ininz a good job 

(B) encourage competition among employees for proemoticns and higher salaries 
(C) emphas'7e improved fringe benefits and concitions of work 

(D) consider employee needs to be more im orcant than organizational needs. 
If a sunervisor should find that he must issue an order his subordinates will 


probably resist, it is advisabie for the supervisor to first 

(A) discuss the order with his subordinates ind give them an opportunity 
to ask questions and make objections 

(B) issue the order without comment and discourage discussion and objections 

his subordinates 

(C) inform his subordinates that he does not agree wit! the order he is going 
to give them, but must carry out the decisions of higher authority 

(0) inform his subordinates that he will take disciplinary action against 
those who cesist carry/ng out the order he will present to them. 


When a supervisor finds that his subordinates differ considerably in the amount of 


attention and guidance they require of him, it would be most advisable for the 
supervisor to 


(A) adjust his supervisory practices according to individual needs 
(=) give an equal amount of attention and guidance to each subordinate 
in order to be fair 
(C) give less responsibility to subordinates who seek reassurance 
(D) permit employees who prefer independence to work strictly on their own. 


Connecting lines on an organization chart represent lines of 
(A) management quality controls (C) authority and responsibility 


(8) work flow (D) fiscal accountability. 


"General supervision’ has been defined as a method in which the supervisor makes 


assignments in broad, general terms and gives considerable autonomy to subordinates, 


in accordance with their knowledge and abilities. A supervisor who uses this 
method is least likely to 


(\) do different work from that of subordinates 

(B) concentrate on long-range problems 

(C) exert excessive pressure on subordinates 

(D) devote considerable effort to training subordinates. 


Of the following, the most important reason why a supervisor should be cautious 
about giving subordinates advice about personal problems is that the 


(A) subordinate may blame him if the advice turns out to be misleading 

(8) supervisor should not discuss personal problems with subordinates on 
office time 

(C) subordinate may lose confidence in his ability to perform on the job 

(D) supervisor may not know enough to give helpful advice. 


Professional or technical consultants may be used most appropriately by a human 
servicos aren tk 


(A irect staff confer+naces centered around programs 
(\) give advice regardior the quality of service or the effectiveness of plans 
(C) supecvise implementation of programs they have developed 
(D) direct the in-service training program. 
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Of the following, the most important aim oa: : les is to assist a 
disadvantaged community to 


(\) accomplish particular tasks within the ser 

(B) evaluate its achievements in the light o! ° ti 

(C) prepare itself to function as an organiz«: 

(0) collect information about the network of } vic 


The goal of the current reorganization of the New Yor: City Vuman Resources 
‘dministration is to provide services in the neivhboriioxws whenever feasible, in 
accordance with the needs of neighborhood residents. f che followinz, a desirable 


result of this would he 


variation in the corbination of services of red in each reighborhood 
uniformity of services offered in each neighborhood 
(C) considerable variation in the amount of community particiostion in 
. each neighborhood 
(D) decrease in the need for utilization of existine community facilities. 


Aprropriate division of functions and resronsibilities betweren tve communities and 
central headquarters is vital to the successful administration of a decentralized 
human services agency in a large urban center. The one of the following which is 
not an anpropriate function to be performed at the cornunitv level is 


(A) review of district plans for policy compliance 

(B) stimulation of maximum use of both new and existing resources 
(C) review of service program goals and relat tonships 

(D) operation of community service teams. 


new approach to the delivery of services currently being developed by the 
New York City Department of Social Services is the Community Social Services Program. 
The one of the following which is not a major feature of this program is 


) stationing of personnel in locations that are easily accessible to clients 
8) decentralization of authority to make decisions regarding services 

) use of purchased services from private organizations to complenent 
directly-delivered services 

(0) preparation of a single annual Iluman Services Plan applicable to all 

Human Resources Districts. 


The Community Social Services Program of the Nepartmert of -oecial Services is ieine 
implemented initially in the followine “uman Resources Districts: 


(A) The Lower Fast Side, Bushwick and Parkchester 

(3) [ast New York, Washington Neights, and the Lower East Side 

(C) Washineton Heights, the Lower Fast Side, and fedford-Ctuvvesant 
(D) South Jamaica, Central Harlem and Villiamsbure. 


Community participation is a key feature of the Community Social Services Program of 
the Department of Social Services. Of the followrine, a correct statement with regard 
to the community participation facet of the program is that it 


(A) extends to the District level the consumer involvement aspect which was 
previously embodied in the client advisory committees of the City's social 
service centers 

(B) implements federal and state regulations which call for community control of 
planning and evaluation of services 

(C) emphasizes participation of local residents in service delivery rather than 
in planning and evaluation 

(D} makes evaluation the responsibility of consumers of social services. 
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hen the citizens of a disadvantaged commu y soir i} ‘ave a positive belief that 

they can work together in solving their problers, and «vs achieve worthwhile goals, 

they are usually said to be working toward a communit' that {s 

(A) complex (EB) diverse (C) funetiancl () underorganized. 
1ich of the following is most likely to produce an itial positive response by 


local residents toward a community project? 


(A) Adoption of the cultural mannerisms and tvynical dress of local residents. 
(RB) Endorsement by a person or an institution trusted bv local residents. 

(C) Solution of a problem for the community to show good faith. 

(D) Promising solutions for all the problems facine the community. 

\f the following, the most common mistake made by participants in social 


controversy is to 


(A) 
(B) 
(C) 
(D) 


Of the following, 


attribute a single point of view of all people of a certain group 
trv to understand all sides of the controversy 

play down the controversy, in the hope that it will disappear 
expect a cooperative resolution of the controversy. 


crisis situation is 


(A) 
(RF) 
(Cc) 
(D) 


stronrer identification with community organizations and institutions 
decreased effectiveness in the operation of community organizations 
more accurate reporting of community events by mass media 

loss of confidence in public agencies and private social services. 


Research in the form of fact finding constitutes a necessary foundation for 


community planning and action. 
a purpose of this type of research is to 


(A) 
(BR) 
(Cc) 
(D) 


"Administrative decentralization 


evaluate services 
determine needs 
establish priorities 
develop a theory. 


reason for emphasizing administrative decentralization would be to 


(A) 
(RB) 
(C) 


(D) 


remove decision-making power from local groups, because recent crises 
have shown that neighborhoods are not yet ready for community control 
provide assurance that district chiefs of City agencies coordinate 
their actions with their central offices 

encourage community participation in those neighborhoods where local 
residents have not taken the initiative to hecome involved in decion- 
making 

provide essential administrative control at the neighborhood level in 


order to give community groups more assurance that their projects can 
be carried out. 
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the most probable result of citizen involvement in a community 


The one of the following which is least likely to be 


gives more authority to Citv agency chiefs 
tistrict level, as contrasted with ‘political decentralization,’ which gives more 


decision-making power to local residents. Of the following, the most important 


38. 


39. 


40. 
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The "outreach" function provides a means of reaching out to disadvantaged 

community residents in order to become familiar with their needs and problems and 

to inform them of services available. Which of the following would probably perform 
this function most effectively in their own neighborhoods? 


(A) community leaders (C) small businessmen 
(B) paraprofessionals (D) school teachers. 


When people of a di: *antaged community are given the opportunity to gain satis- 
faction and self-confidence from small accomplishments, the most probable result is 


(A) discouragement of progress toward major goals 

(B) development of a false sense of achievement 

(C) progress toward overcoming more difficult problems 

(D) development of unrealistic and unachievable ambitions. 


At times it is necessary for a community group to form a temporary work alliance 
with other self-interest community groups for the mutual support of programs. Such 
a temporary work alliance is usually called a 


(A) federation (C) union 
(B) coalition (D) society. 


GO ON TO NENT PAGE 
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Answer questions 41 through 45 solely on the basis of the following paragraph. 


With the generation gap yawning hefore us, it {s well to remember that 20 years ago 
teenagers produced a larger proportion of nonwedlock births than today, and that 
the illegitimacy rate among teenagers is lower than among women in their twenties 
and thirties. In addition, the illegitimacy rate has risen less anong teenagers 
than among older women. 

It is helpful to note the difference between illegitimacy rate and illecitimacy 
ratio. The ratio is the number of illegitimate babies per 1,000 live births. The 
rate is the number of illegitimate births per 1,000 unmarried women of childbearing 
age. The ratio talks about babies, the rate talks about mothers. The ratio is 
useful for planning services, but worse than useless for considering trends, since 
it depends on the age and marital composition of the population, illegitimacy rate, 
and the fertility of married women. For example, the ratio among girls under 18 is 
bound to ‘e high in comparison with older women, since few are married mothers, 
However, the illegitimacy rate is relatively low. 


Of the following, the most suitable title for the foregoing passage would he 


(A) The Generation Gap 

(B) Moral Standards and Teenage Illegitimacy Ratio 

(C) A Comparison of Illegitimacy Rate and Illegitimacy Ratio 
(D) Causes of High Illepitimacy Rates. 


According to the passage, which of the followire statements is correct? 


(A) The illegitimacy rate has fallen among women in their thirties. 
(B) The illegitimacy ratio is the number of illegitimate births 
per 1,000 unmarried women of childbearing age. 
(C) The illegitimacy ratio is partially dependent on the illegitimacy rate. 
(D) The illegitimacy rate is more useful than the ratio for planning services. 


According to the passage, of the following age groups, the illegitimacy Tatio would 
be expected to be highest in comparison with the other grouns for the proup aged 


(A) 17 (B) 21 (C) 25 (DB), 29. 


According to the passage, of the following age rroups, the illegitimacy rate would 
be expected to be lowest in comparison with the other groups for the group aged 


(A) 17 (B) 21 {G)) 25 (D) 29 


As used in the foregoing passage, the underlined word composition means, most nearly, 


(A) essay (B) makeup (C) security () happiness, 


49. 


50. 
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Answer questions 46 through 59 solely on the basis of the following paragraph. 


In counting the poor, the Social Security Administration has developed two poverty 
thresholds that desipnate families as either ‘poor’ or “near poor. The Administra- 
tion assumed that the poor would spend the same proportion of income on food as the 
rest of the population but that, obviously, since their income was smaller, their 
range of selection would be narrower. In the Low Cost Food Plan the amount allocated 
to food from the average expenditure was cut to the minimum that the Agriculture __ 
Department said could still provide American families with an adequate diet. This 
Low Cost Plan was used to characterize the ‘near poor catepory, and an even lower 
Ecoromy Food "lan was used to characterize the "poor caterorv. The Economy Food 
Plan was based on 709 cents a person for food each day, assuming that all food would 
be prepared at lime. The Agriculture Department estimates that onlv about 10 per- 
cent of persons spending 70 cents or less for food each dav actually were able to 
get a nutritionally adequate diet. 


Nf the followin the most suitable title for the foregcine paragraph would be 


(A) The Superiority of the Economy Plan over the Lost Cost Plan 
(B) The Need for a Nutritionally Adequate Diet 

(C) Food Expenditures of the Poor and the Near Poor 

(D) Diet in the United States. 


According to the paragraph, the Social Security Administration assumed, in setting 
its poverty levels. that the poor 


(A) spend a smaller proportion of income for food than the average non-poor 
(8) would not eat in restaurants 

(C) as a group includes only those with a nutritionally inadequate diet 

(9) spend more monev on food than the near poor. 


According to the paragraph, it would he correct to state that the Low Cost Food 
Plan 


(A) is above the minimum set by the Agriculture Department for a nutritionally 
adequate diet 

(8B) gives most people a nutritionally inadequate diet 

(C) is lower than the Economy Food Plan 

(") represents the amount spent bv the "near poor." 


As estimated by the Department of Acriculture, the percentage of people spending 
70 cents or less a day for food who did not ret a nutritionally adequate diet was 


(A) 100: (B) 90% (Cc) 10% (DB) 0%. 


As used in the fcregoing paragraph, the underlined words allocated to mean, most 
nearly, 


(A) offered for (3) assigned to (C) wasted on (D) spent on. 
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1, ihe ratio of unemployed to employed among the employables who moved to Central 
tv in 1961 was 
(1) ft to 3 (B) 1 to 12 (C) 1 to 24 (D) 1 to 36 
n 1972, if the number of employables entering the citv increases to 33,000 and the 

inemployment rate is half the rate of 1971, the number of unem>loyed amonp the 
enployables who move to Central City in 1972 will be 
(A) ? (i) 1,650 


(C) 8,250 (D) 16,500, 


ollowing, tiie vear with the wreatest number of unemployed among erployables 
oved to Central City was 


(B) 1965 (C) 1967 (D) 1971 


‘€ the f 


Ollowing,the period during which the greatest change occurred in the 
numver of unemploved among emplovaliles who moved to Central City was 
1261 and 1962 


(EB) 1963 and 1964 


(C) 1969 and 1970 


(D) 1970 and 1971. 


there are 640 


unemployed 
Central 


among the 23,479 e who mcve to Central City in 1975, 
unemployment rate in 1975 will he most nearlv 
2 (FP) 3% 


City 


tC) 42 
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), The Chairman of the City's Commission on Human Rights has : tated that the purpose 
xf the ethnic survey of City employees conducted recently w.s to determine 
the degree of "ethnic group identity" among City employees belonging 


to va. ous minority groups 
) whether there has been job discrimination by City agencies against 
minority croups 
the basis for the establishment of quotas which would guarantee jobs 
to a certain number of minority group members 
1 method of strengthening the merit system through establishment of 
jiscretionary hiring practices for minority group members. 


ve f the following, an important purpose of the separation of the functions of income 
maintenance and social services in the Department of Social Services is to 


(\) give caseworkers more time to provide services and counselling 
to welfare families and individuals 
(5) eliminate investigation of all families and individuals applying 
for public assistance 
(©) enable the Department of Social Services to implement the State- 
mandated program of check pickup at the New York State Employment Service 
()) facilitate decentralization of the allocation of funds for services. 


98. Changes in the Food Stamp Program made by new regulations passed this spring 
resulted in 


(A) raising both the maximum income for non-public assistance families who 
would he eligible to purchase stamps and the bonus fer families already 
participating, but also raising the cost of the stamps 

(®) lowering both the maximum income for non-public assistance families who 
would be eligible to purchase stamps and the bonus for families already 
participating, but lowering the cost of the stamps 

(C) raising the maximum income for non-public assistance families who would 
be elicible to purchase stamps and lowering the cost of the stamps, but 
lowering the bonus for families already participating 

(9) lowering the maximum income for non-public assistance families who would 
be eligible to purchase stamps and raising the cost of the stamps, but 
also raising the bonus for families already participating. 


59. The Federal Revenue Act of 1971 includes special benefits for working mothers who 
have child-care expenses in that it provides for deduction of 


(A) specific household and child-care expenses for single, divorced or 
widowed parents only 
(B) specific household and childcare expenses for dependents under 15 
(C) carfare and other expenses relevant to the mother's employment 
() allowance for an additional dependent for the person caring for the child. 


60. A serious objection raised by those who consider the proposal for revenue sharing 
bv the Federal government to be a major threat to welfare reform is that 


(4) states and localities would not nave sufficient control over how the 
money is spent 

(8) increased state and local control over expenditures has consis<ently 
worked against disadvantaged and minority groups 

(C) there are no Federal provisions for enforcement of compliance with 
regard to discrimination for reason of race, color or national origin 

(D) the state governors would not be able to set their own priorities or 
monitor violations by local officials. 
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Accordin? to the Yenart 


ct « cJal ‘ervices, tl following, the smallest 
number of recipients wo! i Sistance consists of 
(A) children under 21 (fC) emy ! 
(B) aged adult ()) adults caring for others. 
Responsibility for euali ontrol studis of public assistance clients’ eligibility 
in New York City by means of interview an inal is of case records is being taken 


over by the 


(A) U.S. Department of ‘alth jucation and ‘el ire 

(B) New York State Department of cial Services 

(C) New York City Department of Investieation 

(D) Bureau of Income faintenance of the !'luman Resources Administration. 


The one of the 
Resources 


followiny which is not a current method being used by the Human 


Administration in its efforts to reduce welfare costs is 


(A) improvement of manaverial capability 

(8B) simplification of procedures 

(C) insistence on yreater accounta tility 

(D) .41 investigation of all applicants fur assistance. 


The one of the following which would be most likely to reduce the tendency of the 
unemployed poor to migrate to urban centers is 
(A) increased federal subsidies to the small 
(B) unionization of migrant farm workers 

(C) a national guaranteed income maintenance program 
(D) an increase in the national minimum wage. 


farmer 


According to the most recent U.S. Census Bureau Report, economic and educational 
gains made between 1960 and 1970 by Puerto Ricans living in New York City, as 
compared with the Blacks, were 


(A) considerably greater (Cc) 
(3) considerably smaller 


about the same 
(0) slightly greater. 


Of the following, modern child-care experts vould consider the least desirable 


setting for treatment of most juvenile of fenders while under p.vfessional supervi- 
sion to be 


(A) the family's home \ a foster home 
(B) a group residence (4. a large institution. 


New York State has instituted a program of ‘work relief," which requires all Home 
Relief recipients who are eligible to work, but cannot find jobs, to "work off" 
their welfare checks on non-salaried work Projects. “his approach has met with 
considerable opposition, mainly because those who oppose it believe that the 
recipients 


(A) may take away jobs which were formerly performed by paid employees 
(B) are unskilled and therefore qualified for an extremely limited number 
of work assignments 
(C) are likely to be resentful and inefficient in carrying out their assignments 
(D) will receive no improvement in financial Status and no assurance of stable 
employment,so that they can he removed from the welfare rolls 
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A sienificant aspect of an amendment co the Social Security Act which 


became 
effective July 1, 1972 is that it introduces into the Act a program of 


(A) public service jobs 


(C) medicel assistance to the aged 
(B) income maintenance 


(D) aid to dependent children. 


During the past few years, the U.S. Supreme Court has made a series of key rulings 
that directly affect welfare clients. None of these rulings nas been concerned 
with 


(A) fair hearings (C) home inspections 


(B) state residence requirements (D) the family assistance plan. 


A demonstration program for public assistance recipients in New York City recently 
approved by the U.S. Department of Health, Education, and Welfare which has been 
the subject of constderable controversy is the 


(A) Incentives for Independence Program (Cc) Work Incentive Program 
(B) Haitian Training Project (D) Demonstration Work Project. 


GO ON TO TuE NEXT PAGE 
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two-vear worl tud ry in to educate k income New York City adults for new 
careers iv comrunity seri e is provided by the 
) College f or tud 
(rT) Lle in t t 
(C) ® Pro 5) 
(D) Mannower reer Jevelopment Agency. 
‘uid iac i sistance in social readjustment for youths between the ages of 
l ’ } en brougut into contact with the criminal courts is provided 
thie 
(A) it icc (C) Youth Services Agency 
( eut ' (D) Spofford Juvenile Center. 
Adult paroles nrobationers and former prisoners should be referred to the followin; 
agenev for ‘el curin; vinp Loyment , counselling and such other assistance as 
the i ! 
(A) The Osher ‘-sociaction 
(B) Central ehalilitation Council 
(Cc) (mer ic *deration 
co) § er rene nd Guidance Service. 
The apenes ' issive TLeonomic Neighborhood Development (MEND) is based in 
the follo : rty area: 
(A) East har! (}) Brownsville (C) South Bronx (D) Arverne. 
The acenc ‘“hilization for Youth is based in the following anti-poverty 
wea 
(A) Lovee (C) East Bronx 
coy 1 (D) Bedford-Stuyvesant. 
the foll ‘ © only program .o which narcotics addicts mav he committed 
involuntariil sponsored bv the 
(A) Addicts ‘ices Aceney (C) Mayor's Narcotics Control Council 
3) Healt » Agency (D) Narcotics Addiction Control Commission. 
The one of tii toring wiich is a part of the Human Resources Administration 
fs the 
(A) Agen ild Development 
(B) Commi i uman Rights 
(C) enartricut ental ‘!ealth and “ental Retardation Services 
(D) Vf Fi icf “‘edical Ixaminer. 


All of the followinz departments or agencies are included in the Human Resources 
Administration execent the 


(A) Addictie trices Arency (C) Nepartment of Social Services 
(3) Denartre et nsumer Affairs (D) Youth Services Agency 
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79. he voluntary city-wide ceitrai resource for public and voJuntary social and health 
organizations which foster oint planning and consumer participation in the 
lelivery of health and welfare services and develops social policy is the 
(A) Community Service yviety of New York 

: (8) Coalition for Health and Welfare 
(C) Social Service Exchanms 
(9. Community Council ot reater ‘lew York, 
‘ 
40, lich one oi Che foli ii 1ograms would pive assistance to a minority group 


member who is having difficulty in renting an anartment in a community that is 
reearded as restricted 


(A) lodet iti (C) New Towns 
(Bb) Open Cities (D) Urban Renewal, 
4 
e 
| 
> ‘ 
Page 18 
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It is senerallv considers advisable for an empiovee whe IS a grievance or Is 
lissatisfied with some condition at work to first 
(A) trv to transfer to a job with more favorable coalitions Dx 


(4) express his complaints fully and freely to his sunervisor 
(C) submit his complaint in writine to higher manarerial personnel 
(9) take advantage of the oreanization's formal erievance procedure. 


lost of the jobs subsidized hy the federal eovernnent under the Emergency Employment 
Act have sone to 


(A) workine mothers 

(5) veterans and heads of low-incone households 
(C) low-income collese graduates 

(9) workers with clerical skills. 


The largest number of jobs available in New York City is in the category of 


(A) sales (C) office work 
(RB) manufacturine (D) service. 


The function of the Work Incentive Provram (WIN) is to provide work experience 


and skills training for 


(A) high-school dropouts 

(6) veterans 

(C) handicapped men and women 

(D) heads of families on welfare. 


Of the following, the most important function of employment testing is to 


(A) determine what salarv to pay the applicant 
(3) indicate how applicants react in pressure situations 
' 


(C) provide information about an applica t's aptitudes and abilities 
(0) determine when an employee will be ready for advancement. 
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nh. ‘he Emergency Employment Act of 1971 was designe. + : iti .overnment for 
nersons in the follo 


catecories: 


(A) "hard-core male unemnloved hetween the ates of , HK ? 

(B) ex-addicts and former criminal offenders 

(C) employable female heads of households who are receivine Aid to Dependent 
Children 

(D) veterans, heads households, and others wh ace "imernploved or 
underemp love 


" %f the followinrs, tie most serious concern of admin: traters of fob training 
programs for the unewrleoved and unskilled is t! 
(A) lack of jobs for zraduatre f the programs 
(3) difficulty of recruitine participant: 
(C over-emphasis on ‘third sector" employment procrans 


(D) searcity of qualified teachers. 


46 Joh restructuring con best he jescribed as a method of improvine utillzation of 
~anpower by 


(A) analyzine the tasks performed in a particular job and rearranging them 
in order to assizn simpler tasks to inexnerienced workers and give 
experienced personnel more time for higher-level work 

(B) helping employers *o identify their nanpower needs and to plan trainine 
Programs for che disadvantared in order t et those needs 

(C) onenine jobs that were formerly unavailable to certain groups because 
of discriminatory barriers or statutorv prohibitions 

(D) matching information about an applicant's background, by means of 
electronic data processing, with a suitable joh onening or training 
program. 


ca he function of the manpower program called Rf 


iTAR is to 


(A) provide placement services for Spanish-speakine neople 
(6) give courses in restaurant work and management 
(C) provide skills training for entry-level jobs in the (‘istributive 
trades 
() train enrollees to operate light and heavy trucks and tractor-trailers. 


on, During the past few vears, the U.S. Department of Labor has sponsored a nationwide 
Program to improve the techniques used to match workers with available job 
openings. One system snonsored by the Department of Labor, which is a computerized 
listing of available ernlovment opportunities in major metropolitan areas, is 


called . 
(A) a job referral! file (C) an occupe*.: nal guide 
(B) a job bank (D) an emp) dictionary. 
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ANSWER SHEET COLLECTION: You are requirew to ian! i> your question »voklet, 
answer sheet, and all scratch paper after competing the test. Any candidate 
leaving the room with the question booklet or scrat,. paper in his possession 
will be disqualified. When you have completed the vest, romain seated and 
Summon the inonitor. The monitor will pick up your answer sheet, question 
booklet, and scratch paper. Leave the building quickly and quietly. You 
should make a note of your application number and include this number in any 
future correspondence with the Department of Personnel] with respect to this 
examination. 


The Department of Personnel, in establishing the key answers to this test, re- 
serves the right to determine which of the answers listed for each question is 
to be deemed and accredited as acceptable. The Department of Personnel further 
reserves the right to cancel and annul any question whenever, upon inquiry, it 


deems that none of the listed answers thereto can properly be considered accent- 
able, 


KEY ANSWERS: The rating key for this test will be posted on the Official 
Bulletin Board of the Application Section of the Department of ‘Personnel, 
49 Thomas Street, New York, N.Y. 10013, for a period of three days beginning 
October 25, 1972. 


Notices 


Please do not telephone the Department of Personnel requesting information as to 
the progress of the rating of this examination. Such information will not be 
avallable to candidates. All candidates in this examination will be notified 
individually by mail of the rating they receive when the rating of all candidates 
in the examination has been completed. Notify this Department promptly in writing 
of any change in your address to insure prompt delivery of such notice. 
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J Whe VEL; J) ictivities te determin 
their 5 iE 3 LVeOVI,: VIL, VILE); A 
dele t ic ic il ith ot r ics 
estab] nd approved by the Federal Office of ic Opportunity for ez h 
progra , Ch £2) PRE. IVS VEL NEL. VERE Enec ;¢s Community Corporation 
‘ pertise so that they can become auton us of C.D.A. supervision 
IV, VL, VII, VIII). 
Advi utive rc ¢ vices.=- layout, telephone, equity e 288 
— b ‘ rled f duties and sponsibilities in the line and stef, -yments 
of HRA an! its constituent ogencics (I, IT, III, IV, VI,4VII, VIII). closely 
at sites with officials, construction people, architects, engineers, nity 
repr tot real tate experts to be certain that future need il be 
included in the pl Cf, ER EIS IVE VES VIEL Vitd).. “See up contrac! id sy 
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Services, Division (1, IV, VI, VIII). 
Krowledee ar | ized (There is a Roman Numeral following cac! factor 
which corresponds to an are: of the Test Plan). 
a 
g 1. Responsibiliti:s of H.R.A. Central and its constituent agencies tnd 
relevant outsi:le asencics (1) 
: 2, Familiarity with developments and prcblems in programs affectiny human 
’ resources and the amelioration or elimination of pvoverty. (TI) 
3. Knowledse of cormunity and public relatiens. (TIT) 
4. Skills in decidinr provlens relative to such on-the-job situations as 
office services, payroll, budeet and personnel. (IV) 
5. Ability to supervise, direct and develop staff. (V) 
6. Familiarity with machine, equipment and supply purchase, usaze and 
manavenent inclucin sefety. (VI) F 
‘ie: SHEL S lancnace usere ineludine vocabulary and reading conrrehension, 
8. Ability to perform arithmetic computations and interoret tables, craphs 
an!’ charts. (VTIT) 
’ 
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| ’ -3- ves 
| 
} 
Dut i¢ mad Re nsibilitic 
the statevent 4 tl netic of examination i corr t nd was confir ut 
Mr. Howard tat li, Assist Director of Personne) of the | in Re 1 
Administration 
There are no syc¢cial medical/physical requirenents for this position 
| I { | 
The tHhuman Pesow Administration wishes an opporti t Y jew the notice 
of examination before publication. 
) 


2 ff ” a. / a P ) 
CL gf ¢ fa f7 fi fae AA 


a y 
% Leonard Y/ Rosenberg 


\ 


Sr. Personne] Examiner 


Approve: by: 


4 ce USue ia iUtwwT 


Sol Buchnan, Acting Chief 


Examinine Division S$ 
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Test PLan Examination Nt IBER 1631] 


= f ’ 
' aa f t ' ‘ 
. 
F | er P if 
' ' t° P . . : 
fF} 
“ ‘ 
a a Yr { of TLRLA. Lt9 mstitucene apeuci md other releve 15% 
public { private ¢ iC1E3, 
Tm. Curreat Pevelopucnts and proble fu sha ficld of human services. 10% 


Iv. Jud { 1 Oil job sitertions 4nvolviig office cervices, 20% 
payroli., budeet end versonccl. 
V. Techaseucs of staff developscnt, eupervisiou and ¢odidones. 20% 
VI. Mecains, enufprert and supply purciese, usase and mansypesent 52% 
including safety. 
VII. Lanpuese usese including vouzbulsry and reading couprchension. 10% 
VIII. Aritin.ctic computations aud fates: cetaticn of tables, chatts, 10% 


and gra 


There will be a total of 109 raltdpic-choice questions, a portion of 
which will ba in cor:on with: 
(1) Sensor Hunan Resources Specialict, Vern. No, 1099. 


rar 


(2) Promotion to Sentor Iluman Resovrces Specialist, Exam. No. 1626. 
| 
| 
} 


} 


EXHIBIT 
y U. S. DIST. COURT 


Pd 


SD. OF NM. Y 


; JAN iS7¢ 


etd 


- a * 4 ° 
-> 
/ 
= ep ‘ { P i 

/ 

fi-/ 
tre ' Leonhard Y, Posenbers Approved by foten® Sol Yachran 
parelt ’ i , 
D0 -taeeredmiahsiaabneiietnie nannies bein tnncemainaairnivamnan UAE OF Divintem CHIT oo oe 


CEFF. Cael aS heres tec ceccasniceissaiemestias jac icecbticgaiimnaretintamincaencoda iy, LON a ee 
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[Test Phan Examination NUMBER 2013] 


7 oo } 
t 
. 1 i 
\ ’ ‘ 
P : 
{ a e ne f tt 
1 ' | f ‘ 
t i i iff tf ¢ ‘ ‘ ( ffer 
f 
; ki t 
Bs } of rol i and j 
IT. Current devel ‘ problems in t fic an 
Tj) Cc nity rc] ‘ le Oren tic ] 
1V. Ju il ooh ont 1 JLuntic involvi of fix serv 2u 
pe oll, bi “pol 
V. VC ju of Laff devil t UbCTrVIsic Ww "s 
View Mociing, r nt J SUpely purchace, DFC uid oP s) 
includii Se ly. 
VII. Ls me usace, Le Jud ocehalary and 3 Comps ( Ju 
Vili. Arit SELe cx t 3 ahG MMterrreietion o1 t } i ti, ] 
an? erapiis. 
Lu 
The Will be a toval ef 100 nultie? -choice qt LONG, A portion of 
Watch will be in cornon with: 
(1) Seator Lisen Resources § ScLilist, Mxan. Mo. 1099 
(2) Vroretion to Senior Hur ul Resour Specialict, * », 1626 


(3) Senior Euman Nesourece Spectalise (mT), Dram. No. 16% 


(4) Prosotion tu Supervisi P Vanuan Resources Speeialisc, tran. Re. 133) 


(5) Supervisine tuwean Resources Specialist (YT), Lxan. 


JAN 1974 
ee 
4 a / 
‘ ( 45 on™ P ae \ 
+. fyProved by i) 

OY aca ceeneipeaesdinitiieeaedibincas essamtaes Unit ce Civision Chict Ri ah Sale h 4 Ree po 3 
ty OF FOr AiveclOrie? Esanlmuttons ccooc dnctorccmmnminc mcs ee Lh asiaktaie ace 
4 
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Letter of Edward O. Howard Dated April 9, 1974 


Covincton, Howarp, Hacoo & HOLLAND 
ATTORNEYS aT Law 
if COLUMBUS CIRTLE, NEW YORK, N. Y. 10023 
> DONALD COVINGTON 
EDWARD oO HOWARD 


ABLE AND SRK oOunser 
KENNETH D. HAGOOD c L 4 NEW YORK n 
ALBERT HOLLAND 


3340 


Ap.il 9, 1974 


Paula Omansky, Esq. 
Law Depar’ ment 
Municipal Puilding 
New York, New York 10007 
Re: Williams v. HRA 
Dear Paula: 


We are setting forth herein information with respect to 
the plaintiffs in the above Captii ned action: 


JOHN GOYCO: 
1. Is with the Department of Employment within FRA. 


2. His "in house" title is Acting Senior Case Manager 
(Public Service Career Program Acting Coordinator). 


3. As Acting Senior Case Manager Goyco: 
a. Is directly responsible for }rogram implementation. 


b. Monitors the Educational Component and on-the-job 
training. 


Cc. Evaluates the program components, education and 
on-the-job training. 


d. Maintains files and records for the program. 


€. Submits weekly reports to the Regional Manager and 
to the Public Service Career Program in the Central Office. 


4. Mr. Goyco has not taken any prior examination for an 
HRA position. 


JOHNNIE Mccoy: 


1. Is with the Department of Employment within HRA. 


2 aer "in house" title is Assistant Coordinator of 
Supplies and Equipment. 
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COVINGTON, GRANT, HowarRbD, HAGoop & HOLLAND 


Paula Omansky, Esq. -2- April 9, 1974 


3. Her present job duties are as follows: 

a. Under the supervision of the coordinator, she 
prepares all purchase orders and makes proper distribu- 
tion of same. 


b. Handles all receipt slips for materials delivered. 


Cc. Serves in the capacity of trouble shooter and ex- 
pediting cfficer pertaining to any problem ii procurement. 


d. Handles G.S.A. Selfservice store procurement for 
control office and delegate agencies exclusively. 


e. Processes all invoi and G.S.A. billings for pay- 
ment. 


f. Prepares all G.S.A. multi-use requisitions for 
Central Office as well as the delegate agencies (used for 
obtainipg merchandire from G.S.A. New Jersey warehouses). 
g- Establishes schedule for field t.ips to delegate 
. agencies for proper utilization of government equipment and 
Supplies. 
h. Regulates the messenger service. 


i. Serves as coordinator of supplies and equipment in 
the absence of coordinator. 


4. Ms. McCoy has not taken any previous examinations for 
HRA positions. 


» DOROTHY WILLIAMS: 
1. Is with the Agency of Child Development. 


2. Her "in house" title is Field Services Specialist. 


3. Under the direct supervision £ the Field Services 
Supervisor, the Fi2ld Services Specialist is the direct liason be- 
tween delegate agency staff and parents and central services. The 
Field Services Specialist is responsible for providing technical 
assistance, training and support necessary for on-going quality 
programs. Her specific responsibilities iuclude: 
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a. Developing schedule so that each center is 
visited on a regular basis. 


b. Determining how an agency is fulfilling terms of 
its contract with CDA, utilizing questionnaires develop- 
ed by program specialists. 


c. Evaluating specific program areas with special 
materials developed by program specialists. 


d. Submitting to the Field Services Supervisor pre- 
pared weekly reports of visits to assigned centers and 
special reports on re-xequests including needs identified 
after effective dialogue with staff and parents. 


e. Clarifying and interpreting a’l city-wide Head Start 
policies to insure awareness and compliance; clarifying and 
interpreting all national guidelines to insure parent par- 
| ticipation in all parts of the program. 


f. Aiter consultation with city-wide area representatives 
and the Pa.ent Involvement Coordinator, conducting orienta- 


tion for staff and parents on PAC authorities and respcnsi- 
bilities. 


g. Monitoring PAC elections. 


h. Assisting parents and delegate agency staff in 


develeping positive relationships and effective PAC's at all 
leveis. 


i. Assisting local directors in mobilizing PAC's. 


4. Ms. Williams has taken one prior examination for HRA, 
as if more fully detailed in 1er affidavit dated the 28th day of 
December, 1973. 


We note the suggested dates set forth. in your letter of 
April 5, 1974 with respect to filing of Defendants' Consoliaated 
Brief (April 29) and Plaintiffs' right to file a reply brief (May 6), 
and we agree to same. 


We trust that the infoxmation set forth above with respect 
to the Williams' plaintiffs suffices for your needs in preparing 
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COVINGTON, GRANT, HowaRbD, HAGOoD & HOLLAND 


Paula Omansky, Esq. -4- April 9, 1974 


your brief. If there is anything more which you wish to discuss 
concerning the matter, would you kindly call me at once. 


Sincerely, 


WARD O. HOWARD 
EOH: aw 3 


cc: Hon. Morris E. Lasker 
Jeffrey Mintz, Esq. 
James Jones 
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Defendants’ Stipulated Facts 


IN THE 
UNITED STATES DISTRICT COURT 


For tHe Souruern District or New York 


SS 
{CAPTION OMITTED] 
———————————_—» 

It is hereby stipulated and agreed among the attorneys 
for the parties hereto that the following statistical data 
would be introduced into evidence and that the witnesses, 
if called, would testify as follows: 


HRS (MDT) Open Competitive No. 1095 


A total of 182 took the HRS (MDT) open competitive 
examination. A total of 143 failed. HRA was able to deter- 
mine the ethnic identity of 27 of those who passed the 
exam, and 69 of the failures. 


HRS (MDT) Open Competitive Examination No. 1095 


Passed Failed Total 
Re ste LC ZE) 44 56 
ok ee 8 (47%) 9 17 
Hispanic .................... 7 (30%) 16 23 
ORR tO 27 69 96 
No ethnic info ......... 12 74 86 
39 143 182 
xX 


Rr 


eo 
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SR. HRS (MDT) Open Competitive Exam. No. 1094 


A total of 193 took SR. HRS (MDT) Open Competitive. 
A total of 145 failed. HRA was able to determine ethnie 


identity of 30 of those who passed and 67 of those who 


SR. HRS (MDT) Open Competitive Exam 


failed. 

Passed 
LLC te) a Maaco sae aaa = IS (31%) 
RTE nok oe see DL 5 (33%) 
Hispanic: A228 : 7 (32%) 
CORN GY 5 cree er eae — 
me) 1) U0) 1 eae ease 30 
NO FECOTO .c.c esc: 18 


48 


SR. HRS Promotional Exam No. 1626 


total of 106 persons took the SR HRS (Pro). 
of 44 persons passed; 62 persons failed. 


Failed 


41 
10 
15 


SR. HRS Promotion Exam 


Passe d 


Blas Sea. cspeeee ete 11 (18%) 
hi 7 eer ae 30 (S8S%) 
Pirspame: 25 3 (37%) 
Other ......... valves ~- 

EL oocck 44 


Failed 


i 
oi 


Total 


A total 


Total 
62 


34 
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SR. HRS (Open Competitive) No. 1099 


A total of 683 persons took the SR. HRS (OC). A total 
of 255 passed; 428 failed. HRA was able to identify the 
ethnic origin of 165 of those who passed and the ethnic 
origin of 241 of those who failed. 


SR. HRS Open Competitive Exam No. 1099 


Passed Failed Total 

IAG os eee 56 (26%) 165 221 

(Vb Sn 5 Sey 101 (65%) 54 155 

oe: mn 8 (27%) 22 30 

RNB (total S28 165 241 406 
Ethnie Origin 

Undetermined ...... 90) 187 277 

GC Re 25d 428 683 


HRS Open Competitive Exam No. 1097 


A total of 606 persons took the HRS Open Competitive 
Examination and 200 persons passed. HRA was able to 
identify the ethnic origin of 122 of the passers and the 
ethnic origin of 206 of those who failed. 


ta 
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URS Open Competitive Exam 


Passe ad 

Black ........ Caen er DD (31%) 

WA TERE cesta cai. a.) OGL) 

Phepanie: 222-4. (19%) 
01S) ne eee ] 
Dt FORAY fcc see. Eee 

Ethnie Origin 

Undetermined ...... 7S 
PBOPAN ese 200 


Faile 
120 
56 
29 

l 


206 


200 


406 


HRS Promotional Examination No 


A total of 5S persons took the HRS promotional 


nation, 14 passed. 


rf 


. 1625 


HRS Pror:otional Examination 


Passed 

| 2 Ti Soe See elves OM iS a 

ia Spleen mene ate & L (20%) 
Hispanic 2... 2... — 
14 


Leonarp RosenBers 


Failed 


36 
4 
4 


44 


Total 
175 


Lid 


exami- 


Total 


If called to testify, Leonard S. Rosenberg would testify 


that he followed the same procedures in performing the 


job analyses for the positions of Senior HRS (promotional 


and ope. competitive) and HRS (promotional and open 


competitive) as was testified to in the Jones case for the 
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position of Supervising HRS. The results of his job anal- 
yses fer these positions are set forth in the respective 
“Field Audit Reports,” which are denominated Defendants’ 
Exhibits M-1 and M-2. 


HELENE V"LULINGHAM 


If called to testify, Helene Wiiingham would testify that 
she followed the same procedures as testified to in the 
Jones case when preparing the examinations listed below 
which are herewith introduced as Defendants’ Exhibits: 


Human Resources Specialist (ARS) Occupationo! Series 
} ! 


Exam. ba abit 

Position No. Ne 
Senior HRS promotional ..... Ree Ores 1626 N-1 
Senior HRS open competitive ... L099 N-2 
ERS promotional ....0.0.0.0000.0..... Tere 1097 N3 
HRS open competitive 2.00000... 1625 N-4 
Senior HRS (MDT) open competitive L094 N-2 
HRS (MDT) open competitive .......... 1095 N-3 


Harotp YourMAN 


If called to testify, Harold Yourman would testify that 
the examination for the position of Senior HES, and for 
HRS, are job related in that they test for knowledge of 
areas of employment within the Human Resources Admin- 
istration to which persons in the titles could be assigned. 

Dr. Mildred Katzell, if called, would testify, as follows: 

The statements made about the Supervising HRS tests 
are equally applicable here. Leonard Rosenberg prepared 
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the job or task analyses which lay the foundation for a 
valid job-related examination in a thorough and competent 


m ‘ner, consistent with professional standards in the field 
0 yyment testing. He gathered data, by interviewing 
persous presently holding the positions of Senior HRS and 


HRS, observed persons performing their work and con- 
firmed and clarified his observations by interviewing per- 
sons within the Human Resources Administration in 
superior and supe>+‘sing positions. 

From the particy.ar tasks observed and learned through 
interviews, he generalized to basic “knowledge and skills 
required,” related each task to the general area of knowl- 
edge to be t ‘ted. Each area of knowledge was “weighted” 
in terms o1 frequency and/or importance, i.e., the test 
plan. 

The resultant products, the job or task analyses them- 
selves, were impressive in their adequacy, thcroughness 
and completeness and provided more information than Dr. 
Katzell was accustomed to finding when asked to work on 
tests based on job analyses. 

As for the examinations themselves, the questions related 
to the test plan; the test plan related to the knowledge and 
skills required; the knowledge and skills required related 
to the job audit of tasks observed and learned about 
through interviews. To t..is extent, the test is job-related 
and performance on the test could be expected to relate to 
performance on the job. 

The item analyses showed that the questions were at the 
appropriate level of difficulty. Persons who did better on 
individual questions did better on the test as a whole. 
Further, the item analyses showed that virtually all of the 
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options were working. While there was one question, which 
100% of the high scoring group answered correctly, never 
did 100° of the low scoring group answer a question cor- 
rectly. There were very few questions where less than 
90% of high scoring group answered correctly. The ques- 
tions cover a full range o: difficulty, which is desirable 
when it is necessary to rank individuals. 

Based on examination of the job analyses, the tests, and, 
the item analyses, these tests do a better than average job 
of covering the requisite knowledge and skills identified by 
the job analyses. 

Dr. KE. Belvin Williams, if called to testify, would testify 
as follows: 


He would answer the question whether the examinations 
for Senior HRS and HRS are job-related in the affirmative 
and with a high degree of confidence. 

He interviewed 10 individuals presently occupying the 
position of Senior HRS or HRS, all hom took, and 
some of whom successfully passed, the challenged examina- 
tions. Each of the incumbents was interviewed at his or 
her work site for approximately an hour. Those inter- 
viewed were of the opinion that Dr. Williams’ purpose in 
talking with them was to conduct a desk audit of the posi- 
tions they presently oceupied. 

Dr. Williams based his opinion on the job-relatedness of 
the tests by utilizing direst and indirect indicators. 

As to the direct indicators, Dr. Williams asked, in the 
course of the interview, for each of the “knowledges and 
skills” listed in the job-analyses, whether these were needed 
or relevant for the successful performance of the indi- 
vidual’s duties. He further asked for and was shown mate- 
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rials, such as reports, proposals, budgetary summaries, 
telephone logs, and schedules of meetings, which docu- 
mented and confirmed the oral description given by the 
respondent in describing the relevance of these knowledges 
and skills. 

On the basis of these “direct” indicators, he concluded 
that the seven test categories named are substantially 
related to the performance level of the positions. One 
converse of his conclusion is that the individual’s lack of 
the knowledge or skill labeled in the test plan would seri- 
ously handicap the effective performance of the duties 
described. ; 

This conclusion is not to be construed to mean that each 
category identified in the test plans is related to each job 
or set of functions in the same way or to the same degree. 
Nor does he claim that each question selected for the test 
was the best or sole representative of any given category. 

Dr. Williams also resorts | to indirect indicators in form- 
ing his opinion. He recorded information in the following 
areas. 

1) The highest level of education achieved. 

2) The number of years worked, either in HRA or an- 
other City agency. 

3) The ease and skill with which the respondents could 
make known and understood by him the intricacies 
of their jobs. 


4) The communication skills of the respondent, e.g., 
precision of expression in describing governmental 
processes, 
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Jn the basis of these indirect indicators, he would assert 
two similar but quite distinct things: 


a) The more educationally qualified and the more 
work-experienced (in HRA work) individuals are 
acceding to higher levels of job responsibility. 


b) The individuals who have advanced to hi, her levels 
of job responsibility tended to pass the test with 
greater frequency than those who had not achieved 
such on-the-job success. The job advances preceded 
the administration of the test. 


Dr. Williams’ evaluation of the indirect indicators sup- 
ported his opinion based on the direct indicators that the 
tests were job-related, 

Dr. Williams did not address himself to related inquiries 
such as the predictive efficiency of the tests, the magnitude 
of measurement error, and the like. However, the mcre 
narrowly stated question of whether the tests were job 
related can be answered in the affirmative and with a high 
degree of confidence, 


Nal 


KOOKLITS 


LEAVING 
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‘ Ae oe ah {rer 
’ kL. & 4 5 2. 71 
( } J me Allo g lis 
} ‘ t os lety ¢ rivuted. READ THIS * ONLY. 
, Yuu é 11 WOrH. 
‘ “¢ . St , . ° ¢ 4 
e { - + 
‘ Lous ¢é i co 4 s 
wou i f é t 1 ¢ » Suton 
the 1 3 » Noy ex r i illowed on 
eccount of lateness { ti lerved includes 
ti for fingerprinting. 
this test baokiet should consist of TE consecetively numbered 
pas ibere are. @) questioas, ali of equvel weight. 
UES? £ » Exer rour booklet the secoad 
is given to be su ¢t defective ja any wey. 


& cor .lete Lioklet. 


information concer: 


protests appears ac the end of the rest booklet. 


If you read theage instructions carefully. you will not need 
to make inquiries, No monitor is auihorized to chauge these 
instructions or the teras an? conditions of this test. You 
will be DISQUALIYIED tf you obtain or attempt to obtain in- 
fornation cr help from books or other sources, cony or at- 
tempt to copy or read the work of another candidate, permit 
your work to be copted or read by another candidates, con- 
Mmunicate fi h ancther candidate, leave your seat 
withot* pe he monitor, sign your naae or put 

any icentif: ansver sheet, or disobey an in- 
struction of You are prohibited from having on 
your desk or in your possession ducing the course of the 
test any reference materials, uotes, or newspapers which 

you h:ve brought with you. 


< 


an essamis 


eanor for aryone to impersonate a candidate 
or permit himself to be impersonated et an examination. 


Saokiny ds NOT permitted. 


Duriug the test you may not leave your seat excent when neces- 
Sary, cad then only when accompanied by a menitor. You ma 
withdraw wichout compicting this test and not have your tcst 
rated oy writing “Ll withdeaw” and your signature on your an- 
Swer sheet ond stving it to the monitcr. tlowever, no canii- 


date wiil bo permitted to depart before 10:30 a.m, 


NOT OSEN THIS BOOKLET WIT I'L THE Six) SIGNAL Vie GIVEN 


° * 
% 
Vv 
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bi! 3 FON A RING ¢ "t6'7 
A 1} rt , > oF lL Ar aa f f { nG 
, wry pa 
IILY RAT 
E each question, Ject thet ar rv 1 you helisve tot i tal e 
an t e liste nd note ti capital tter ne’ ve «ce er 
nen, on the of ficial er eet, £4 the 3 t ¢ tion 
aud record your r by bl ening 2 n tl li ifately 
to the right of t capital Jetter represenci Our vOY, ny a No. 2 
pencil. 
When recording your ansvwet th a No. 2 pancil, f111 in ic ent spoce between th 
dotted lines and the full leasth of the lines Ly procs i on the fr ila 
using several strokes. 


IF YOU W1Sii TO CH E AN ANSVER, ERASE THE OLD ANSWE 
out. ‘hen record your new anayver in the proper sp 


OMPLETFLY. Do not cross it 


Do not make any morks except your answer, lore than one answct to a question or 
improperly erased answers or stray marks c.used by: resting the peucil on the paper 
may be counted as wrong ancwers. 


Use pen (blue or black ink) for filling in the headings on yours enswer sheet but use 
only the No. 2 pencil for recording your answers. 


Do not fold the answer sheet or damace the edges because the rating is done by 
machine. 


Since there are only 4 answers listed for each question, the spaces to the right of 
the letter E on the anewer sheet are not used for this examination. 


To help you understand the procedure, the following sarple item is given: 
SAMPLE 9: The sum of 5 and 3 is 
(A) 2 (B) 8 (Cc) 9 (D) 11. 


The sum of 5 and 3 is 8, so thet the acceptable answer is shown thus on your answer 
sheet: 


SAMPLE 0: A szs22 R emeerene C s2z<s2= D 


CONTINUE ON 
FOLLOWING 


6 PACE 
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Final Key Answers for Written Test 
Held GCctorer 14, 1972 


he following are the finul Ley answers as adopted by the Commission at a meeting 
held on - These key ansi 3 pesult from careful con- 


sideration of all protests suomitted by candidates. 


a) 
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Cs) Ss ri re : t 74 1} ‘ 
(3) supervisor hes ¢ be coployve's uaconsel tive 
brin up the pie 
‘isfy ‘ : S ee € weber ropore the rrobl 
(uD) uper ov has jscussed ti reve th his supers t 
Of the follewiac, the situetion ir which a « €rvisor would have te e tl erent 
effort in order to communicate effecrivel with his suo: te WOX cccur 
(A) there is 4 larve gap betrveen the Supervisor's icl und and e€ f 
and that of his suh Linate 
(E) the subortinete ve alré arned about the inforuation throurh 
inforral channe)s 
(C) the sul.ordinat we coupleted their education much were recently 
than the suservi r 
(D) the surervisor ha een ywitt Mizgation for a much shortcyr 
tine than t of his subora 
Of the following, the factor which vould be wost critical in influencing whether 


subordinates accept or resent a Supervisor's authority is the 

(A) manner in which rhe supervisor uses his authority 

(8) frequency with which the supervisor ignores minor viol«tions of rules 
(C) degree of delegation to suboidinates by the Supervisor 

(D) cultural attitudes of indivi 


#1 subordinates towsi7 authority. 


In which one of the following situations would employces be most likely 


) V 
temporarily difficult worki conditions without excessive complainine? 


(A) The organization has a strict policy of disciplinary action against 
uncooperative employees. 


(B) Employzes do not have the right to take part in "job actions" or strikes 

(C) An. rospucere of mutual trust and good human relations exists betwee 
subordinates and managerial personnel. 

(D) Relatiorshinos between Subordinates and managerial personnel are strictly 
businesslike. 


Assume that an agency tos + rganized into integrated work teams. Instead of 
assigning empioyees performing the same task to a single unit, such as a typine 


pool, those performing different but interdependent parts of an activity are put 
into the sare we-k group. Of the foliowing, the most probable result of such a 
reorganization would be to 

(A) permit more efficiert work scheduling (C) decrease training costs 

(B) achieve greater cconomvy (D) improve employee job satisfaction, 


The need for identification with a work group has bee’ found to be one of the mest 
powerful on-the-job metivetions. Of the following, the enployee who is leas 
to have a strong attachuent to nis work group is one who 


(A) is at the very bot of the erganization's promotional ladde 

(LE) belones to a smal artmenc 

(C) works with others cf simila; buckeround and interest 

(D) has worhea for the Organiza: ion for a e-nsiderable perted of tino, 
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/ lit t 1 23 ent experts, the one of the follewiny situations which 
world be ther t sirni ent ineic fon that « loyees of an organization are 
dissatisfied with their supervisors and feel thst they are being treated unfairly 
is one in which 
(A) emplovees submit a large number of wort-related suerestions 
(i}) many empicyces gre unproductive and seem to be contin illy 

loafing on the job 
(C) union m rship has recently increased 


(D) turnover is low in spite of a comparatively good Jabor market. 


A supervisor who } 


ias informal, friendly relationships with his subordinates is 
conducting himself : 


(A) appropriately; good informal relationships set the stare for 
better comsunication between the supervisor and subordinates 
on work-related problems 

(BD) inappro; fately; subordinates who have infor al relationships with 
their supervisor are not likely to accept h. authority 

(C) approp 
his su 


lately; friendly relationships between the supervisor and 

»ordinates will create a true feeling of ecuality between them 

(D) inappropriately; subordinates are likely to becone suspicious of 
insincerity and fearful of heins maninulated. 


Specizlization is a commonly-used method of increasing productivity and efficiency 
in a large crganization. Task specialization means that separate and comparatively 
simple parts of a more complex jo. are performed by different employees. Of the 
followins, this type of specialization probably would not 


(A) reduce training costs 

(B) permit the use of more specialized equipment 

(C) simplify the develoynent of job controls 

(D) give most emplc-ees a greater sense of accomplishment. 


Some management experts who have studied inforra] commu. ication patterns jn large 
Organizations believe that the office grapevine is an effective means of communication, 
Of the following, an importaat function perforned by the grapevine is to 


(A) permit feedback and spread information faster than most formal 
communication svstems 


(B) give employees important information from reliable sources 
(C) permit management to identify rumor-mongers and troublemakers 
(D) bring informal leaders to the attention of management. 


Recent studies of morale and productivity tend to show that 


(A) the corr ion between morale and productivity is rather low 
(P) hipk moral s associated with high productivity 
(C) low morale . associated with high productivity ‘ 


(D0) lew morale is associated with low productivity. 


Research studies have indicated that teamwork among employces is most likely to 
result in highey pruductivity in a situation where 


(A) emp. yees accept as legitimate management's demands for higher productivity 

(3B) management strongly encourayes the workers’ competitive spirit 

(C) employees are unified for the purpose of protecting themselves against 
managemoat's demands 

(D) management does not encourage employees to make independent decisions. 
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The relationship between boredom on the job and fatipue 1s correctly stated ag 
follows: 

(A) boredom usually results in increased fatigue. 

(B) A worker usually beconzes bored when he expends a mindiuwum of 


physical energy. 
(C) Fatigue usually results in boredom. 
(D) A worker who is bored does not usually become fatigued, 


The "halo effect" can properly be suspected of harming supervisor-subordinate 
relationships when the supervisor 


(A) does not discriminate between the good and poor work of an employee 


considered by him to be genera .y superior 
(B) expects his subordinates to 1 it him in an impersonal and forza] 
manner 


(C) hesitates to discipline employees because of an extreme need for 
them to like him 

(D) is unable to gain his employces' confidence because he cannot shed his 
reputation for being hardboiled and unfair. 


’ 


It is generally considered that the best interview is the one in which the 
interviewer talks less than the person interviewed. The one of the following 
which is an effective device to encourage the other person to talk during the 
interview is for the interviewer to . 


(A) summarize the feelings the person has expressed, omitting details 
and incidentals 
(B) keep silent and show no indication of hts reaction to what the nerson 
is saying 
(C) clearly show his approval or disapproval of what the person is saying 
(D) talk to the person in terms of concepts rather than specifics, 


Studies of groups of workers doing the same job under the same conditions have 
shown that there are always a few workers who have nore accidents than the rest. 
The one of the following which is least likely to be a finding of such studies 
is that those who have the most accidents probably are 


(A) middle-aged (C) inexperienced at the job 
(B) poorly adjusted to work (D) less efficient than other workers. 


Work measurement has been defined as "the determina. on of the proper amount of time 
and effort required for the effective performance of a specific task." Of the 
following, the factor which would be least relevant in stu ying an operation by 
mea~» of work measurement is whether the operation is 


(A) repetitive with constant standards of quality 
(B) compensated for at a prevailine rate of pay 

(C) routine in nature and relatively easy to perform 
(D) performed in large volume. 


Of the following, "participative management" can be defined best as a method in 
which 


\A) subordinates have formed groups for the purpose of gaining participation 
in the decisions of manapement 

(B) management makes a practice of encourasing subordinates 3s a group to 
discuss and participate in decisions on a wide variety of work-related 
problems 

(C) managerial employecs are given varied assignments on a rotating hasis 

(D) management gives ail emplovees the opportunity to participate in mafor 
policy decisions. 
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tui resl Ca out t or coh PETE Die i. te them 
hen t yr finds tht his su linat aif fe lerably in the amecunt of 
att t ; idavawe the require o Ho, At o nost advisable for the 
supervisor to 
(4) adjust hie supervisory practices acenrdine te indivitual needs 
()) give an equal anount of attention ant guidaoce to each subordinate 

in order i f 
(C) vive less ility to subordinates wie seek reassurence 
(D) permit ¢ prefer independence to work strictly on their own. 
Connecting lines on an organization chart represent lines of 
(A) managenent quality controls (C) avthority and rcsponsibility 
(S) work flow (D) fisc#) accountability. | 
"General supervision’ h Leen defined as a method in which the supervisor makes j 
assirncre:ts in broad, al terns and gives considerable autonomy to subordinates, 
in eccordance with thet knowledge and abilities. A supervisor who uses this j 


method is lesst likely to 


(4) de different work from that of subordinates 

(B) concentrate on lonpg-renge problens 

(C) exert excessive pressure on subordinates 

(D) devote considerable effort to trainine subordinates. 


Of the followine, the mest important reason whv a supervisor should be cautious 
is that the 


about giving subordinates advice about personal] problers 


(A) subordinate may bla him if the advice turns ont to be misleacing 

(3) supervisor should not discuss personal problems with subordinates on 
cffice time 

(C) subordinete ray loses confidence in his ability to perform on the job 

(D) supervisor may not know enough co give helpful advice. 


Professional or technical consultants may be used most appropriately by a humin 


(A) Gireet staff con°ersnevs centered around procrans 


slity of service or the cifectiveness of plans 


( 

\r) 

(C) svoervise innle mt mm of prosrams they have developed 
(2) \ 
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YE the following, the most drpoctane atm « mit ‘ it 26 to-aeshiet 4 
di Ivant ed cormunit to 
‘ 
acco-rlish particular tacks within the servicer 
evaluate Its ach'’evements Jn the liehr or or hs "OALE \ 
(C). prerare itself te funetion an orgarive:, ‘ entit \ 
(D) collect information avout the network of soci. Ger sbes 


The g0al of the current reorganization of the New Yor! (ftv Vurnan "esour 
Administration is te Pp vide services in the neich! mre whit rs feagi le, 
accorcance with the needs of neighborhood recidents. Of the folleving, a desirable 


result of this would be 


(A) variacion in the conbination of services offered in each neds srhood 

(8) uniformity of services offered in each neighborhood 

(C) e¢snsiderabl. varlation in the eaount of community particivation in 
each neightorhood 

(2) decrease in the need for utilization of existing cerivunity facilities. 


Appropriate division of functions and responsibilities between the communities and 
centrol bexdquarters is vital to the successful administration of a decentralized 
human services agency in a large urban center. The one of the follovring «ich is 
not an anprepriate function to be perforned at the community level is 


(A) revies of district plans for policy conpliance 

(DB) stimulstion of maximum use of both new and existing resources 
(©) reviers of service program goals and relationships 

(D) operition of community service teams. 


A new appreich to the delivery of services currently beinc developed bv the 
New York City Department of Social Services is the Community Social Services Prorram. 
he one of the following which is not a major feature of this progran is 


(A) stationing of personnel in locations that are easilv accessible to clients 
(2) decentralization of authority to make decisions rerarding services 
(C) use of purchased services from private organizations to complenent 
cirect!v-delivered services 
(0) preparation of a single annual Human Services Plan applicable to all 
Human Resources Districts. . 


The Community Social Services Program of the Departrert of <-ocial “1 ciees ts -eing 
implenented initially in the followine "uman Resources Districts: - 


(A) The Lover East Side, Bushwick and Parkchester 

(®) Nast “ew York, Washington Heights, and the Lower "ast Side 

(C) Vasiineton Neights, the Lower Fast Side, and fFedford-“tuvvesint 
(D) Sonth ‘amaica, Central Harlem and Williamsbure. 


Community participation is a key feature of the Community Social Services "rogram of 
the Department of Social Services. Of the following, a correct statenent vith repard 
to the community participation facet of the program is that it 


(A) extends to the District level the consumer involvement aspect wnicn - is 
previously embodied in the client advisory comuaittees of the Citv's social 
service centers 

(B) implements federal and stat2 regulations which call for comrunity e-ntrol of 
planning and evaluation of services 

(C) emphasizes participation of local residents in servic: delivery rather than 
in planning and evaluation 

(D) makes evaluation the responsibility of consumers of social services, 
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t , ref Iveani od ce 1 v ‘ Ju & bed) thet 
¢ te r in sol theity ¢ ; ' achle mrtiral sate, 
Ly to he wo lr t ' ' ! { 
(A) ex (L) diverse (CY. frie ’ (3b) underore vile 
feh of the fol] ’ is nos Ivy tay 1 t i} positive respon by 
J al faent towal1a Co } rol { 
(A) Ado c t liucal ? nit i) cys xf losal restdents 
(%) Endor t b persou o7 t “1 t ted local re tents 
(C) o] c a proule for th ce it gsoud faith. 
(D) Proms 1 solutions for all the proble faci t!} ¢ nECYs. 


Of the following, the most common nistake wade by participants in social 
controversy is to 


(4) attribute a sinele point of view of all poovle of a certait froup 
(B) try to understand ail sides of the cont 


(C) play down the controversy, in the hope that it will dissppear 
(D) expect a cocperative resolution of the contreversy 
Of the follovine, the most probable result of citizen involvement in a community 


crisis situation is 


(A) stronper identification with community orgarizations and institutions 
(Rh) decreased effectiveness in the oneration of community orpanizaticns 
(C) more accurate reporting of community events by mass media 

(i); loss of confidence in public ayencies and private social services. 


Research in the form of fact finding constitutes a necessary foundation fo 
community planniag and action. The one of the following which is least likely to be 
a purpose of this type of research is to cy aa 


(A) evaluate services 
(B) determine ne -ds 

(C) ertabi'sh priorities 
(D) develo» a theory 


"Administrative decentralization’ gives more authority to City agency chiefs at the 
(district level, as contrasted with ‘political decentralization,” which gives more 
dccioion-making power to local residents, Of the t»iiowing, the most important 
reason for emphasizing administratiy> decentralization would be to . 


(A) remove decision-making power from local groups, because recent crises 
have shown that neighborhoods are not yet ready for community control 

(8) provide assurance that district chiefe of City agencies coordinate 
thedr actions with their central offices : 

(C) encourare community particdpation in those neighborhoods where local 
residents have not taken the inittative to become involved in decion- 
mal.ing, 

(D) provide esnential administrative control at the nejghborhood level in 
order to give community groups more asgurance that their projects can 
be carried out. 
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The "outreach" function provides a means of reo disadvantarved 

community residents in ord: to become famili reeds and prehlens and 

to inform them of services available. Which vould probably perforn 
thie function most effectively in their own n Lghborhoods? 

(A) community leaders (C) small businessmen 

(B) paraprofessionals () school teachers. 


When people of a disadvantaged community are given the opportunity to vain satis- 
P t & 
faction and self-confidence from small accomplishmence, the most probable result is 


(A) discouragenent of progress .toward major gonl 
(B) development of a false sense of achicvement 
(C) progress toward overcoming more difficult problens 
(D) development of unrealistic and unachievable ambition 


9) 


At times it is necessary for a community group to form a temporary work alliance 
with other self-interest community groups for the mutual support ¢f programs. Such 
a temporary work alliance is usually called a 


(A) federation (C) union 
(B) coalition (D) society. 
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45. As used in the foregoing passage, the underlined word demeaning means, mst nearly, 
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Answer questions 41 Lineugh 46 eelcly on the baste ef the follewiis pevrage. 
~— be 


Too often in the part saeiety recanted the ¢xisting soelal weliere premes ce, 
preferriup tu tin’ with refIncenents when fundsnental vofors ves ty cyvser. Vi 
been a Gevmontne. depradiza welfare sycten In which the iistroncut of geverrveen 
was wrengluidy ane o7 tly ured. it has ben a sysrec ' rich has oaly alseanated 
those forced to bens.at 1t10a Lt aad demoralized those vho had to eun:nister 1t at 
the level where the pain was cleeriy visit! 

tlou on a course in which cash benefits, providing 

rf 


There is a need to put this nz 
¢ 


basic level of support, sre nfovred in such a way ae to intrude es Little es 


c 


possible into privacy and celf-respect. It is difficuli to deftue a bsale level of 


support, no matter how high or low it might be set. In the end, however, the 


decision in not determined so much by how much is truly adequate fc: a femily to 


meet all of its nerds, but by th. resources evailabice to carry out the proniee. 
That may be a harsh fact of life but it ic also just thet - a fact of life. 


Of the following, the mort suitable title for the foregoing pessege would be 


(A) The Need for Government Control of Welfare 

(=) Determining the basic Level cf Suppere 

(C) The Need for Velfare Reke. 

(D) The Elimination of Welfare Prograus. ' 


In this passage, the author's greatest criticism of the welfare system is that it 


is too 


(A) disrespectful of recipients 
(LE) expensive to aduinister 

(C) limited by regulations 

(D) widespread in application. 


According to the passage, the basic level of support is actually detersined by 


(A) how much is required for a fanily to meet all of its needs 
(B) the ase of the recipients 

(C) how difficult it is to adisinister the program 

(D) the economic resources of the nation. 


In this passage, the author does not argue for 


(A) a work incentive system 
(B) a basic level of support 
(C) cash benefits 

(D) the privacy of recipients. 


(A) ineffective (B) exnensive (C) overburdened (D) humiliating. 


46. As used in the foregoing passege, the underlined word ine; tly means, most nearly, 


(A) foolishly (B) unsuccessfully  (C) unskillfully (D) unhappily, 


47. 
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50. According to the paragraph, which of the following is included in the subenployment 
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Ansver questions’ 47 through 29 selely on the baste of th following poravranh, 
The unemployment rete, which counts the une loyed di. t} sense that they ore 
actively looking for vork and unsble to find 4%, gives a relatively cucerfied 

idex of c¢ fe conditions in unity A better ind is tl vig it 
rate which 1icludcs the wiensloyment rece and eleo inclutes these vorkiag part-rin 
while they are trying to get full- tine work: those head of househelcs under 65 


years of age who earn less than $60 per week worbine full-time, and ticse individual: 
under 65 who are not heads of houscholds and earn less than $56 per w 

time job; and an estimate of the males " 

in ghetto areas, 


in a fuli- 
not counted," which is a very real concern 


Of the following, the most suitable title for the foregoing paragraph would be 
(A) Employment in the Uriied States 

(5) Part-time Workers and the [Ceorony 

(C) The Labor Market anc the Conmunity 

(D) Two Indicators of Economic Conditions. 


On the basis of the paragraph, which of the following statements is coirect? 


(A) The unempioyment rate includes everyone who is not fullv employed. 

(B) The svubeuploynent rate is hivher than the unemployment rate. 

(C) The unemployment rate gives a more complete picture of the economic 
Situation than the subeuployment rate. 

(D) The subemployment rate indicates how many part-time workers are 
dissatisfied with the number of hours they work per week. 


As used in the foregoing paragraph, the underlined word superficial means most 
nearly 


(A) exaggerated (B) official (C) surface (D) current. 


‘< 


rate? 


(A) Everyone who is unemployed. 
(B) All part-time workers. 
(C) Everyone inder 65 who earns less than $56 per week in a fu)l-timé job. 
(D) All heads of households who earn less than $60 per week in a full-time job. 
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Anever questions’ 51 throush 55 selely on the bests of the dlagram presented below: 


HO THE INNE-CiTy FAMITY Til MRPANVILLE SPENDS ITS MONEY 


A. Food 

BR. Shelter 

C. Clothing 

D. Household Operation 
E. Medical Care 

F. Transportation 


G. Miscellaneous 


51. According to the diagram, the perceitage spent on medical care is 


(A) 7% (B) 8% (C) 162% (D) 18% 


552. According to the eiagram, the total percentage spent on food, shelter, and 
clothing is 


(A): 55% (B) 60% (C) 70% (D) 75% 


“33. 'In a typical period, if the family spent $30 on transportation how much did it 


‘spent on household operation? 


((A) $120 (B) $192 (C) $300 (D) $480 


"54. If the family income is $125 a week, how much does it spend on transportation each 
‘year? 


((A) $60 (B) $2690 (C) $600 (D) $2,600 


55. Assume that the annual income of a family was $5,409 for several years. Last year, 
the income went up 5%, and the family then tripled the typical percentage for 
‘household operation. The amount spent on this item last year was most nearly 


((A) $891 (B) $1,134 (C) $1,296 (D) $2,721 
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Answer questions 56 thre 3 folely on the baets of the ¢t le presented belo 
| 
AUDC Far Nes E ! 
Referred to and Enrolled in WI‘! Program, 1469-197) 


Child, are 


56. According to the table, how many AFDC family members were referred to WIN in 1971? 


(A) 1.792 (B) 2,004 (C) 2,388 (D) 2,478 


57. According to the table, the number of AFDC children, 16 and over, who were enrolled 
in WIN in 1971, was greater than ii.2 number enrolled in 1969 by 


(A) 12 (B) 34 (Cc) 38 (D) 52 


58. According to the table, the number of AFDC mothers who were enrolled in 197] 
increased over the number enrolled in 1969 most nearly by 


(A) 20% (B) 322% P (C) 54% (D) 837 


59. In 1972, if the number of AFDC mothers referred to WIN increases 5% over 1971, the 
number of fathers referred increases 8% over 1971, and the number of children 
referred increases 5% over 1971, the total number of AFDC family members that will 
be referred in 1972 is most nearly 


(A) 1424 (B) 1524 (C) 3130 (D) 3990 
| 60. According to the table, the percentage of AFDC fathers not enrolled in WIN in 1971 


of the number referred that year Is most nearly 


(A) 23% (B) 25% (c) 712 (D) 77% 
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H @ OF 23 follerric which acewinta for ti largest purtien of the budget of 
the | uw Retcsources Avrinictration ts 

(A) person } ad e Sor 

(BD) public ; Monee andl medfeaid 

(C) services toa childs ind youth 

(D) commontty organiszaisen and development, 
Accercing to the latest statistles pubjiished by the U.S. Department of Health, 
Ecucation and Veliatve, the state which spent the largest anount of money per person 
for pullic assistance jis 

(A) California (E) Massachusetts (C) Pennsylvania (D) New York. 


According to the most recent U.S. Census Bureau Report, the group living in 


Nicw York City which has tha lowest income level ie tia 


(A) Blacks (B) Puerto Ricans (C) Dowinicans (D) Haitiens. 


The group that contains the largest number of individuals receiving public assistance 
is: 


(A) children under working «ge (C) the aged, diratied and bl? id 
(B) unemployed heads of fauilies (D) unemployed single persons. 


A major difficulty faced by new arrivals to cities since 1950, which did not exist 
for earlier European immigrants, is the fact that the majority of present-day 
arrivals 


(A) must forfeit thei. netive culture patterns 

(B) have an obviously darker skin color than most long-time residents 
(C) have little education 

(D) have few occupational skills. 


Generally speaking, low-income persons do not make maximum use of opportunities and 
services available to them, mainly because 


(A) most paraprofessional workers, while sincere in the desire to serve, 
are unable to reach the hard core 

(B) much of the routine paperwork in public assistance programs is now 
assigned to paraprofessional workers 

(C)’ they have become incre. singly self-reliant and prefer to cope with 
their problems without help 

(D) they lacl. the confidence and know-how necessary to make their needs 
known to the proper persons or agencies. 


The one of the following problems which has most recently become a serious concern 
of the Youth Service Agency is the 


(A) increasing high school drop-out rate 
(B) resurgence of fighting youth gangs 
(C) spread of youth narcotics addiction 
(D) lack of recreation programs. 


Of the following, the most recent development with regard to welfare recipients is 


(A) introduction of the declaration of need instead of an investigation of 
eligibility 

(B) a major emphasis on employment progr.ms 

(C) increased use of casework therapy ani psychiatric counselling 

(D) acceptance of narcutics addicts for home relief. 
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70. A be {ce ch] tive of the proposal icy revenue oe} acing curr r) under coi iderellona 
by the U.S. ress is to provide 
(A) estate 1 local pov t JLél vw sour: ef rev fren th 
feceral rove rrnent cic] erect mtrol over hew this revenue Js snent 
(B) the fedevel poverr C wits preater control over spe ng of certain 
federaljy-raised tex reve 
(C) safepus: againet Jopreper elloesttion of funds by strate ard lecal 
officials aud incentives to stetes for reporting violztious by 
local gov unene 
(D) a method oi sharing federal reve cue with the states and localities 
in accorcance with tieir requir penditures for public assi nce 
and social welfare corvices. 
| 
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f r fy eh ef tl fclle ryviee avi thy t Ie e 4 } 
rvir or | C ist e ¢ not l o! ts art of the cli ? 

(A) Hone manas 0 (C) Child protectton 

(6) Fandly ~liantas (D) Educetionrai cor:.selling 


The one of the following which js not part of the organizaticn of the Youth Services 
Agency is the 


(A) Neighberheod Youth Corps (C) Youth Develepzzent Centers 
(B Kecreatlon Support Fropgran D) Youth Vocational Training Program. 
re 


The one of the following which is not included under the recently organized Office 
of Special Services for Adults in the Department of Social Services is 


(A) senior citizen programs (C) foster care for the aged and disabled 
(B) Medicare (D) adult inetitutions. 


Several juvenile decention centers formerly run by the New York State Office of 
Probation are now opercted by the 


(A) Youth Services Agency (C) Office of Child Development 
(B) Department of Social Services (D) Judicial Conference. 


Decentralized administration oi the City's anti-poverty efforts is provided by the 


(A) Cormunity Corporations (C) Human Resources District Councils 
(B) Counc’l Against Poverty (D) Neighborhoo. Youth Corps. 


~ 
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(A) recor t ef et for the ¢ end t \ P 
aud he ayenc 
( cent ch riztwitaire a ce 1 index ec ’ fs and 
iid Kt t rest th 
(Cc) ccnter ich pr infor { pout ' errals to {71 
and health agenci wd proprietary nursine he : 
(0) confiden lal advisor, service to help potential contributors ev ste 
local voluutary health and wellare apenctes. 
Which one of the follovins is an important purpose of the formation of the Of! 
of Special Services for Children in the Department of Social Servicc 


(A) Greater progremmctic ‘integration of the protective anc gupportive service 
to children who are abused, neglected, depencent, Celinquent er in nee 
of services. 

(B) More professional attention to clild abuse cases and prorot ecurt action 
to penalize parents of abused or nevlected children. 

(C) Separatien of prozrams and facilities for children adjudged to be 
delinquent from special se -!e-s for other dependent, ebused or neglected 
children. 

(D) Inereased attention to heme-finding and foster car: and adoption 
services rather than instituifeoncl care for dene: dent children. 


The one of the following which is provided by th> Depertment of Socia) Services 
for current, former and potential public assistance recipients only is 


(A) information services (C) referral serviccs 
(B) child welfare services (D) homemaker servicor. 


A major foal of the Department of Secial Services which is part of the current 
reorganization - d the separation of income maintenance from social services is 
to 


(A) limit the provision of public social services to those New Yorkers 
who are eligible for pub]‘c assistance 

(B) make public social services available to all New Yorkers, whether or 
mot they reyuire financial assistance 

(C) refer clients wno require social services to private arencies wherever 
possible 

(D) emphasize casework treatment and referral of clients for psychiatric 
services rather thar programs to effect environmental change. 


Of the following, the main functions of the Manpower and Career Developnent Agen 
(MCDA) of the Human Resources Administration are to 


(A) train the unskilled, upg-ade existing skills, develop job oprorturities, 
and place newly-trained people in jobs 

(B) operate manpover, recruitment and testine centers under contract with 
private organizations 

(C) provide remedial education and follow-up for disadvantaged potentis 
college studeuts and vocational testing end counsellins for veteran 
and ex-a2ddicts " 

(D) provide job development, interviewine and placemert, and manpows 
research services. 


a 
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Wats : t f tot * 2 t face of ¢ ‘ lope. 
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Prote 3} } e + ae legib] Il’ tyenevricten. Each protest must 
begin é rf er of qui 1 being pretested,. 
Prot } peein by ladicatrin ! naver t ndicat considers correct, 
and should py ni rea 1 erpe it wi Clearl: identify any quoted source: 
1 Dp et | ' t ti Gate ' state his address. 
otest tted 2 2 c Les be considered. 
After the Department of Perse has considered all protests, properly filed, it 
will ne a fii dete ti er¢ and will thereafter post the final key on 
the Officia tic Bulletin of the plication Section of the Department 
of ler for a period of thi AWS Such posting constitutes official pub] 
notice bs rtnent of Pers } to all candidates of the final determination 
in tnis connection. 
The t of Person » in establJishiny kev answers to this test, reserves 


the determine whi of the wers listed for each question is to be 
dec and ac lited as acceptable, snd whether more than one of the answers 
listed for each question is to be deemed ncceptable and accredited as such. The 
* 


t reserves the right to cancel and annul any question whenever, 
upon jn ers that none of the listed answers thereto can properly be 
s | 


consicer 


of Personnel requesting information as to the 
e available 
c examinaticn will be notified individually 
the rating thev receive when the rating of all candidates in the 
examination has been completed. Notify this department prowptly in writing of any 


‘ange dn your address to ist 


don. Such information will not 


re prompt deiivery of such notice 


° 
229a 
EXHIBIT N-2 ANNEXED TO 
DEFENDANTS’ STIPULATED FACTS 
HT aes | te 
Cit of or! 
Nep i & % 
EX/ WILON NO, 2 - Ort 1 ( ) ‘ATIC R 
SUNTO¥ RLY } 
EXAIN NATION NO. L094 = OF COMPTTLTIN EGietia STICN FOR 
SENIOR RUMAN PESOULCES. SPECIA CMANE EL AND Th y ) 
Yate: October 14, 1972 r Totel Tine Allowed yarn i 


Fill in your application nusber, room nu ber, ar 


BELLS: First Pell; Test booklet given out. RAD THIS PAGE ONLY. 

Second soll: Turn the page and begin wort. 
1 End cf test. Stop all work. This bell is runp 2's hour 
after the secon’ bell. If you finish before, raise you 
hand to let the monitor know 


TEST BOOXLET: This test boskliet has both tests and there should be 19 pages. 


All questions ste of equal weight. Check your test booklat 


to be sure it has all the pages and is 


SENTOR HUMAN ROSOURCLS 

TONS 1-80 

NO. 1094 - 0.C. SENIOR HUMAN RESOURCES SPECIALIST (i9T): 
ANSWER QUESTIONS 1-70 AND bi-°O 


OR: If you are taking two or more tests follow the special instructions 
yeu have been given concerning questions to answer end time allowed. 


You must hand and 


rea the 


sheet, on which you may make a record of your ansvers 
reference. 


WARNING: Smoking is NOT allowed. You will be disqualified 1£ you copy 
answers from anvone or use books or notes. 


It is against the lew to take the test for somechody else or to 
let somebody else take the test for you. 


This test booklet is the property of the Department of Personnel 
and you must turn it in ef the end of the test. It is worth 
more than $250. You will be subject to penalty if you do not 
turn it in. 


After the test starts, nobedy can leave till after 10:39 a.m. 

, ? 
Nobody can come in after 10:30 a.m, If you went to diop out of 
the test and do not want to have vour answers rated, 


"Y withdraw’ on your answer sheet and sign your nam 


DO NGT TURN THE PAG? UNTY 
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TI , ont f ficial sheet, Find the yr ¢ i the s e as the questic ) 
a cord you r inrz the space een the dotted lin i edicte] 
to the right of the capital letter representir ir selected answer, usi: a Ne. 2 
} 
pencil. 
hen recording your anewer with a No. 2 pencil, ftl1 in the entire space between the 
dotted lines and the full Jength of tie lines ty pressing firmly om the peacil and 
using several stroxes. 
IF YOU WISH C} AM A Rs SE F OLD ANSWER COMPLETELY. Do not cross it 
out. Then record your answer in tt proper spac 
marks except your answer. More than one answer to a question or 
ed svers or st I + caused by’ resting the pencil on the paper 
as wrong answers. 


Use pen (blue or black in for filling in the headings on your answer sheet but use 
neil for record your answers. 


Do not feld th answer sheet or damage the edges because the rating is d 
machine. 


Since there are only 4 answers listed fer each quest 1, the spaces to the right of 
the letter E on te answer sheet are ncot used for this examination. 


To help you understand the procedurc, the following sample item is given: 
SAMPLE 0: The sum of 5 and 3 is 
(A) 2 (B) 8 (Cc) 9 (pb): 21. 


The sum of 5 and 3 is 8, so that the acceptable answer is shown thus on your answer 
sheet: 


SAMPLE 0: A ===== Ro ensccmne: C s=ss== D =s=== 
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8 A soe. 62. D 
9. D 36. D 63. 8 
10, 4 f 37. D 64. A 
ll. A 36. B 65. B 
12. A 39. C 66. D 
13. A 40. B 67. B 
14. A 41, C 68. B 
15. A 42.. A 69. C 
16. A 43. D 7c. A 
17. B 44. A fix 3B 
18 B 45. D ven D 
19. A 46. © ia. 9 
20, A 47. D 74. B 
21. A 48, B 75. 2B 
22. C 49, C 76. C 
23. C 50. C a, ® 
24. =D Sl. B 78. =A 
25. B $2. C 73. =A 
26. C 53. & 89. D 
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22 C 47. D 82 B 
23 Cc 48. 3B 83 D 
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h t time for @# supervisor to give advice about a job-rel 
problem which din has brought up durtns an interview is u )) ifter the 
(A) subordinate has told hia all the fact 
(8) supervisor has determined the « »loyee's unconsclous motives for 

bringing, up the prot. 


(C) employee tb submi 


i writ? port on the problem 
(D) supervisor has d? sed th } uw with his superior. 


Of the followirg, the situation in which a supervisor 


would have to make the rreatest 
effort in order to cummunic 


e effectively with his sub rdinates would occur when 


(A) there is a large fap between 
and that of his suberdin:te 

(B) the subordinates 
informal ch 

(C) the subord tes have completed tl 
than the supervisor 

(D) the supervicor has been with 
time than most of his 


the supervisor's background and experience 


Ss 
have already learned abou- the information through 
xynels 


ieir education much more recently 


cr 


he organization for a much shorter 


Of the following, the factor which would be 


most cri il in influencing whether 
subordinates accept or resent a superv 


isor's authority is the 


(A) manner in which the Supervisor uses his authority 

(B) frequency with which the supervisor ignores minor violations of rules he 
(C) dezree of delegation to subordinates by the surervicor 

(D) cultural attitudes of individual subordinates toward authority. 


In which one of the followine situations would employees be most likely to accept 

temporarily difficult working conditions without excessive conplaining? 

(A) The organization has a strict policy of disciplin 
uncooperative employees. 


(8) Employees do net have the right to take part in "job actions" or strike 

(C) An atmosphere of mutual trust arid good human relations exists 
subordinates and managerial personnel. 

(D) Relationships between subordinates 
businesslike. 


ary action apainst 


S. 


between 


and managerial personnel are strictly 


Assume that an agency has been reorganized into integrated work te 
assigning employees performing the same task to a 
pool, those performing different 
into the same work group. 


reorganization would be to 


ams. Instead of 
single unit, such as a typing 
but interdependent Parts of an activity are put 
Of the following, the most probable result of such a 


(A) permit more efficient work scheduling 


(C) decrease training costs 
(B) achieve creater economy 


(D) improve employee job satisfaction. 


The need for identification with e 


2 wor! group has been found to be one 
powerful on-the-job motivations. Of the following, the employee who is 


to have a strong attacliment to h.s work group is one who 


of the most 
least likely 


(A) is at the very bottom of the organization's 
(8B) belongs to a small department 
(C) works with others of similar backeround and 


promotional ladder 


interests 
(D) has worked for the Organization for a considerable period of time. 


ayy 


10, 


tk. 
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According, to fany mana nt ex , the one of the following situctions whic! 
la be the rost Ienificant j cation that ennlo es of on ore mn Gi 
di tisfied with their “visor aid feel t! they e bein; ited unfsirl 
i¢ one a v ‘ 
(A) empl é hmit a Jarge rumber of vork-related su ations 
())) nany ¢ loyées ar 1 Oductive and seem to be continually 
loafing on the jol 
CC) x n menvership ! recently increased 
()) turnover is low i1 jte of a comparatively good labor market. 


A supervisor ssho has informal, friendly 


relationships with his 
conducting himself 


(A) appropristely; 


better corrmunic 


good informel relationships set the store for 
ition between the supervisor and subord: nates 

on work-related problems 

inappropriately; su! 

their surervisor art 

approprictely; 


ordinates who have infornal relationships with 
likely to accept his euthority 
friendly relationships between the and 


his subordinates «ill create a true feeling of equality lLetween 


supervisor 


them 


propriately 


inap ; subordinates are likely to become suspicious of 
insincerity and fearful of being manipulated. 


Specialization is a commonly-used method of increasing productivity and efficiency 
in a large organization. Task specialization means that separate and 
Simple parts of a more complex job are performed by different employees. 
following, this type of specialization probably would not 


comparatively 
Of the 


reduce training 
bh) permit the use of more specialized equipment 
C) simplify the development of job controls 

give most 


costs 


employees a greater sense of accomplishment. 


€ 


Some studied informal communication patterns fn large 
organizations believe that the office grapevine is an effective means of communication. 
Of the following, an important function performed by the grapevine is to 


management experts who have 


(A) permit feedback and spread information faster than most formal 
' communicati:n systens 

give employees important information from reliable sources 
permit manarement to identify rumor-mongers and troublemakers 
bring informal leaders to the attention of management. 


(P) 
(C) 
(D) 


Recent studies of morale and productivity tend to show that 


(A) 
(P) 
(C) 
(D) 


the correlation between morale and productivity is rather low 
high morale is associated with high productivity 

low morale is associated with high productivity 

low morale is associated with low productivity. 


Research studies have indicated that teamwork among employees is most likely to 
result in higher productivity in a situaticen where 


(A) 
(3) 
(C) 


emplovees accept as legitimate manarcement's demands for higher productivity 
strongly encourages the workers' competitive spirit 

employees are unified for the purpose of protecting themselves against 
management's demands 

Management does not encourage employees to make independent decisions. 


management 


(D) 


~ 
= 


15. 


16. 


if. 


18. 
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(C ' ©. revult n redo 
(D A rker whe red doc not u } beconue faticued 
The halo effect" can eriv be suspected of larmine cu rvisor-subordinate 
relat lips when the uperviso 
(A) foes not discriminate between the cood and poor work of an employee 


considered by hiv to be generally surerior 
(B) expects his subordinates to treat hin in an impersonal and formal 


manvwer 
(C) hesitates te discipline employees because of an extreme need for 
them to lite iim 


(D) is unable to gain his ennleyees' confidence hecause he cannot shed his 


reputaticn for being hardbotled and uufair. 


It is penerally considered that the best interview is the one in which the 
intervicwer talks less than the person interviewed. The one of the followinp 
which is an effective device to ercourape the other person to talk during the 
interview is for the interviewer to 


(A) summarize the feelings the person has expressed, omitting details 
and incidentals 
(B) keep silent and show no indication of his reaction to what the person 
is saying 
(C) cleariy show his approval or disapproval of what the person is saying 
(D) talk to the person in terms of concepts rather than specifics. 


Studies of groups of workers doing che same job under the same conditions have 
shown that there are always a few workers who have more accidents than the rest. 
The one of the following which is least likely to be a finding of such studies 
is that those who have the most aecidents probably are 


(A) middle-aged (C) inexperienced at the job 
(B) poorly adjusted to work (D) less efficient than other workers. 


Work measurement has been defined as "the determination of the proper amount of time 
and effort required for the effective performance of a specific task." Of the 
following, the factor which would be least relevant in studying an operation by 
means of work measurement is whether the operation is 


(A) repecitive with constant standards of quality 
(PB) compensated for at a prevailing rate of pay 

(C) routine in nature and relatively easy to perform 
(D) performed in large volume. 


Of the following, "participative management" can be defined best as a method in 
which 


(A) subordinates have formed groups for the purpose of gaining participation 
in the decisions of management 

(B) management makes a practice of encouraging subordinates as a group to 
discuss and participate in decisions on a wide varlety of work-related 
problems 

(C) manarerial erployees are piven varied assienments ona rotating hasis 

(D) management gives all employees the opportunity to participate in major 
policy decisions. 


hr 
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t k qu t niake obfectic 
(i) i ie the or itbout comment and ¢Ciscoura:e discu 1 and objectior 

by his whordinat 
(Co). az his iuordinates that he do not ¢ t ith t orecr he is going 

to sive th but st carry out the decision of r suthorit 
(D) inform his suberdinates that he wi take d linir et i ainst 

tl e who resist carryins out the order he will present to them. 
When a supervisor finds that his subordinates differ considerably in the amount of 
attention and puidance tney require of him, it would be most advisable for the 
supervisor to 
(A) adjust his s rvisory practices -ording to individua] needs 
(i) gi é l anount of attention an! guidance to each subordinate 

in er to be fair 
(C) give less respensibility to subordinates who see! reassurance 
(D) permit employees who prefer independence to work strictly on their own. 
Connecting lines on an organization chart represent lines of 
(A) manarement quality controls (C) authority and responsibility 
(3) 4 flew (D) fiscal accountability. 
"General ervisior has been defined as a method in which the supervisor makes 


assipnn in broad, general terms and gives considerable autonomy to subordinates 
in accordance with their knowledge and abilities. A supervisor who uses this 
methed is least Jikelyv to 


(A) do different work from that of subordinates 

(B) concentrate on long-range problems 

(C) exert excessive pressure on subordinates 

(D) devote considerable effort to trainine subordinates. 


Of the following, the most important reason why a supervisor should be cautious 
about giving subordinates advice about personal problems is that the 


(A) subordinate may blame him if the advice tusns out to be misleading 

(B) supervisor shculd not discuss personal problems with subordinates on 
office time > 

(C) subordinate ray lose confidence in his ability to perform on the job 

(D) supervisor may not know enough to give helpful advice. 


Professional or t«chnical consultants may be used most appropriately by a human 
services agency to 


(A) direct staff conferences centered around programs 

(P) give advice regardiny the quality of service or the effectiveness of plans 
(C) supervise implementation of programs they have developed 

(D) direct the in-service training program 
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28. Appropriate division’ of functions and onetl ‘ t ¢ c vd 
central he ters is vital to the ful inist: Ota r 
human services avency in a larze urban center. 22 one f 
not an appropriate function to be performed at the con unity le es 


(A) review of district plans for policy compl tance 
(2 stimulation of maximum use of both new and existi 


4 


urce 


D 


review of service prosram poals and relationship 
operation of community service teams. 


29. <A new approach to. the delivery of services currently beine develoned 
New York City Department of Social Services is the Com: ty Social Services Program. 
The one of the following which is not a major feature of this procram is 


(A) stationing of personnel in locations that are easily accessible to clients 
(B) decentralization of authority to make decisions regarding service: 
(C) use of purchased services from private organizations to complement, 
directly-delivered services 
(D) preparation of a single annual Iluman Services Plan applicable to all 
Human Resources Districts. 2 


; . ; , =v . ners 
30. The Community Social Services Program of the Department of Social Services is leing 


implemented initially in the following "uman Resources Districts: 


(A) The Lower Fast Side, Pushwick and Parkchester 

(B) East New York, Washington Heights, and the Lower East Side 

(C) Washington Heights, the Lower Last Side, and Fedford-Stuyvesant 
(D) South Jamaica, Central Harlem and Williamsburg. 


31. Community participation is a key feature of the Community Social Services lrorpram of 
the Department of Social Services. Of the following, a correct statement with repard 
to the community participation facet of the program is that it 


(A) extends to the District level the consumer involvement aspect which wes 
previously embodied in the client advisory com. ittees of the City's social 
service centers 

(B) implements federal and scate regulations which call for community control of 
planning and evaluation of services 

(C) emphasises participation of local rcsideats in service delivery rather than 
Lit plonuing and evaluation 


D, makes eva.uation the responsibilit of consumers of social servic: 
A P y 


36. 


37. 
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Of the fol! the most commo fet made by 7 tici its in social 
c trov 1 t 
(A) attr it sinc] oirnt of view of @1l people of a ccrtain grou; 
(B) trv t understar ‘11 sides of the controversy 
(C) play dom the cert versy, in the hone that it vill disappe< 
(D) expect cooperative resolution of the contreversy. 
inc, the most probable result of citizen involvement in a community 
ti 1 is 


A) stronger identification with community orpanizations and institutions 
) decreased effectiveness in the operation of conmunity organizations 

) more accurate reporting of community events by mass media 

1) loss of confidence in public agencies and private social services. 


Research in the form of fact finding constitutes a necessary foundation for 


community planning and action. The one of the following which is least likely to be 


a purpose of this type of research is to 


(A) evaluate services 
(RB) determine needs 

(C) establish priorities 
(D) develop a theory. 


"Administrative decentralization’ gives more authority to Citv agency chiefs 
‘district level, as contrasted with |‘ political decentralization,’ which gives 


at the 
more 


decision-making power to local residents. Of the following, the most important 


eason for emnhesizing administrative decentralization would he to 


(A) remove decision-making power from local groups, becase recent crises 
have shown that neighborhoods are not yet ready for community control 

(8) provide assurance that district chiefs of City arencies coordinate 
their actions with their central offices 

(C) encourage community participation in those neighborhoods where local 
residents have not taken the initiative te become involved in decion- 
making 

(0) provide essential ad 
order to give corm 
be carried out, 


ninistrative control at the neighborhood level in 
nity groups more assurance that tieir projects cai 


40 
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en people of a disadvantaged cormunity cre eiven the opnorturity to 
faction ant self-confidence frow small aceon: lighncuts, t) st m 
) discouragement of vrogress toward major roals 
b) development of a false sense of schievenent 

) progress toward overcoming more difficelt preble 

) developaent of unrealistic aud unachievable aubitions. 


n 


At times it is necessary for a cemaunity group te form a temporary worl) 
h other s2lf-interest community groups for the mutual support of pro 
2 temporary work sallicnce is usually called a 


(A) fedcraticn (C) union 
(B) coalition (D) society 


GO ON TO NEXT PAGE 
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© much by how h is truly adequate for a family to 
f a t br ft resourc atlabhle to curry out the promise. 
Th ma 2 t of ul but it i 0 just thut - a fact of life. 
41, Of the folle:rin the most suitable title for the foregoing passage would be 
(A) The ed for ¢ rnmnent Con ol of € re 
(B) Det 1 the ‘sic Level Support 
(C) The ed for l Refor 
(D) The ti f Welfare Progra 
42. In this paccacc, the author's greatest criticism of the welfare s¥stem is that it 
is too 
(A) disresnectful of recipients 
(B) expensive to administer 
(C) ldmited by regulations 
(D) widespread in application. 


43. According to the passap 


(A) 
(3) 
(Cc) 
(D) 


44, 


(A) 
(B) 
(C) 
(D) 


, the basic level of support is actually determined by 


how much 1s required for a family to meet all of its needs 
the cge of the recipients 

how difficult it is to administer the pregram 

the economic resources of the nation. 


In this passage, the author does not argue for 


a work incentive 
a basic level of 
cash benefits 

the privacy of recipients. 


system 
support 


45. As used in the foregoing passage, the underlined word demeaning means, most nearly, 


(A) 


ineffective (B) expensive (C) overburdened (D) humiliating. 


46. As used in the foregoing passage, the underlined word ineptly means, most nearl 
Pp g& , quunee -_ y, 


(A) 


foolishly (B) unsuccessfully (Cc) unskillfully 


(D) unhappily. 


59. 
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(3) The t rate is higher than the uner 
¢€¢} The nt rate pives a more complete 
Situation than the subennloyment rate, 
(D) The subenmployment rat {ndicates he many 
dissatisfied with the number of hours they wor 


s used in the foregoing paragraph, thi 


(A) exareerated (B) official 


According to the paragraph, which of th 


) Lvervone who is unemnloved. 
) All part-time 
) Evervone u 

) All 


workers. 
65 who earns less 


heads of 
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than $56 


households who earn less 


Td 


(Cc) 
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superficial 


(D) 


is included 
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current. 
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Accordi to the diapra t percentage snent on medical care 
(A) 7 (B) 83 (C) 163 (D) 
Accordi to the diagram, the total percentage spent on food, 
clothings 
(A) 55% (B) 60% (C) 70% (D) 
In a typical period, if the family spent 
spent on household operation? ; 
(A) $120 (B) $192 (Cc) »300 (D) 
If the family income is $125 a week 
year? 
(A) $60 (B) $260 (C) $600 
Assume that the annual income of a family was $5,499 for several vears. 
the income went up 5%, and the family then trinled the tvpical 
household operation 


(B) $1,134 


») 
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j 


18%, 


shelter, 


75% 


$48 


percentage 


6” 


Yas 


0 


(D) $2,600 


721 


Transportation 


and 


$30 on transportation how much did it 


how much does it spend on transportation each 


Last year, 
for 


The amount spent on this item last year was most nearly 


243a 
Exhibit N-2 Annezxed to Defendants’ Stipulated Facts 
ANT uv! seetior “H throusi. f Solely on ¢} basis of the tab} presented be 
AF] {] ( in | ] 
Referred to {1 Enrolted in WIN Prop 1969-197) 


i a 


Enrolled 


| 
ee Dn eT 
ne as tray paar ae 
keer er 
iahanaaiadlass'S 57 vane se eas 
16 and over 


56. Accordine to the table, how many AFDC family members were referred to WIN in 1971? 


(A) 1,792 (B) 2,004 (C) 2,388 (D) 2 


2,478 


57. According. tc the table, the number of AFDC children, 16 and over, who were enrolled 
in WIN in 14°1, was greater than the number enrolled in 1969 by 


(A) 12 (B) 34 (Cc) 38 (D) 52 


58. Accordine to the table, the number of AFDC mothers who were énrolled in 1971 
increased over the number enrolled in 1969 most nearly hy 


(A) 20% (B) 32% (C) 54% (D) 83% 


59. In 1972, if the number of AFDC mothers referred to WIN increases 5% over 1971. the 
number of fathers referred increases Sx over 1971, and the number of childr 
referred increases 5% over 1971, the total number of AFDC family members t 
be referred in 1972 is most nearlv 


(A) 1424 (RB) 1524 (Cc) 3130 (D) 3990 
69. According to the table, the percentage of AFNC fathers not enrolled in WIN in 197 


of the numher referred that year is most nearly 


(A) 23% (B) 25% CC} 71 CD) 77% 
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Services Agency is the 
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(D) Haitians. 


did not exi 
of present-day 


since 1959, which 
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long-time residents 
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t 


assistance 


naximum use of opportunities and 


in the desire to serve, 
tance programs is now 
with 


ry to make their needs 


serious 


an investigation of 


counselling 


concern 


follovine, the most recent development with regard to welfare recipients is 


to ; 
' . 
i 
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ne] 
hject 
c ¢ 
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(A) state and 


Jeral p 


(B) the 
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11 sove rn 


245a 
Exhibit N-2 Annexed to Defendants’ Stipulated Facts 
ent ce on by ; eral court, veeular report 
¢ ffi pich u c} ra) Col wor ¢ ne 
ents ¢ 
Nisabled (C) 3 to Ne € ( 
{ (D) “oedicaid 
ive cf the proposal for re iu ring currently und 
! to provide 
loc v] FOvernments 3 it} n urce of rev Wwe f t! 
vernment and preater control over he re i 
pov ment with pr ter control over spnendine of ex: 
bast Se tax revenue 
against ir oper allocation of fi ! state and Lc 
and incentives to states for renoy tine vi tions by 
ent 
of sharing federal reven with the tates and ] 


with the 


D 


- 


required expenditures for public assis 


GO ON TO NEXT PAGE 


r cc 
e 
rtc 


~ 


~w 


76, 


Rk 


78. 


~ 


&0. 


246a 7 
Exhibit N-2 Annexed to Defendants’ Stipwated Facts 


: ae r the ' : t 
7 ( l n Jj 

i utd 1 Devel t (ny 

( ¢ t (D) fhe Office of kartly ay Care. 
Vide : Lit on f ‘ in t fullowing poverty erea 

ys 1 C (C) East New Yor! 
(Fr) (D) Bedf Stuyvesant. 

( < 8 calle : ACT base LT the follovy ins? rty Tres 
(A) Wi famsbu (C) East Varler 
(i) Central Karle (D) Lower st: Si . 
Cf the foltovine, the organization which hae waged an aperressive campaifnn to inform 
the poor of their les 21 rip > is the 

A Aid Society (C) American Civil Liberties Union 


National Welfare Rights Orpanization (D) National Urban League. 


Which one of the following prosrams is designed to mect the need for paraprofessionals 


in nt agencies and institutions? 
(A) Concentrated Imployment Program (C) Opportunities Industrialization Center 
(ff) Public Service Careers Program (D) Work Incentive Program. 


The popular name for the program authorized by Title I of the Demonstration Cities 
and ‘fetropolitan "evelopment Act of 1960 is the Federal 


1ti-Poverty Prosram (C) Model Cities Proeranm 
fanpower Development and Training Prorram (D Revenue Sharing Program. 
§ > , 


An agency which currently sponsors sunpo.cive vrograms to rehabilitate former 
narcotics addicts through full-time paid employment in such fields as park 
maintenance, rodent control, and messénger services is the 


(A) Volunteer Prison League (C) Judicial Council 
(B) Vera Institute of Justice (D) Daytop Village. 


Project “Manhood Foundation can best be described as an organization which provides 


(A) manpower services for former convicts aged 16 and over 

(8) supportive counselling for former narcotics addicts 

(C) remediai education and vocational training for juvenile offenders 
(D) skills traininp for the disabled. 


The correct description of the functions of ASPIRA is that the organization provides 
disadvantaged youth of Puerto Rican background with 


(A) educational counselling, financial assistance and career planning services 
B) vocational counselling and tiaining in the skilled trades 

(C) remedial. rehabilitative and recreational services 

(D) remedial education and vocational placement services. 


Which of the following is a federal residential preeram that offers work experience, 
education, and skills training for disadvantaged youth from 16 to 21 years of are? 


(A) Trainine Resources for Youth (C) Vocational Foundation Incorporated 
(8) Youth Imployment Service (D) Job Corps. 
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sidized hy the federal povern t uncer the Emergenc Dloy:22 


2ads of low-income heiuseholds 
low-income college graduates 


clerical skills. 


The largest number of jobs available in New York City is in the caterory of 
(A) sales (C) office worl 
(B) manufacturine (D) service. 

G4. The function of the Work Incentive Prorran (WIN) is to provide work experience 


and skills training for 


(A) high-school dropouts 

(RB) veterens 

(C) handicapped men and women 

(D) heads of families on welfare. 


. Of the following, the most important function of employment testing is to 


(A) determine what salary to pay the applicant 

(B) indicate how applicants react in pressure situations 

(C) provide information about an applicant's aptitudes and abilities 
(D) determine when an employee will be ready for advancemeiit. 
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( t r eT 1 ies which are filled by migrant ers 
t foll.w er programs,the one which coes not receive feceral funds 
is thie 
(A) Joh Corps (C) Work Incentive Prosrar 
(3B). 1 2 rhood Youth Corps (D) Emergency Placement Prorram. 
The Wonderlic Personnel Test is a widely used standardized test which 
Predict employmeut success. A study of an experimental group of disa 
vouth who were successfully emploved after participation in a training prorr 


this sroup's scores on the test were considerably lower than those 


listed in the test manual as minimum passing scores for particular occunations. 
Of the followine, the most approprtate conclusion that can be drawn from these 
results is the 


(A) test is not a valid test for the experimental group 

q iment was yoorly conducted and should be repeated under improved 
tions 

(C) youths may actually be in need of additional training 

(D) test is not useful in predictine on-the-job success for anv croup. 


£9. Joh restructuring can provide new careers for the disadvantaged by 


A 


(A) creacine entry-level positions and meanineful promotion opportunities 
for inexperienced persons : 

(B) eliminating the more difficult duties of a job 

(C) lowerine the educational qualifications for hirher-level jobs 

(D) training individuals in those aspects of a job in which they are not 

skilled. 


90. The New York Plan is designed to provide jobs and training for minority grqup 
members in the 


(A) buildings and construction trades 

(B) garment and needle trades 

(C) printing and publishing trades 

(D) waterfront and warchousing trades. ° 
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l ( sheet, 1] ratch pey ite he et. Any fdrt 
le ij t I = 4 ne - , ( ry booklet o Daye in his p ession 
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I ict } ratch paper. Leave the butldine cugjekly and ot ietlv. You 
sheuld ake a note of your apolicati number ’ Include this number in any 
futu spendence with the Dery artment it Personnel with respect to this 
ex 


The Department of Persenne)], in establishing the key answers to this tes 


t tc, re 
Serves the right to determine which of tie anrwer Jistel fer each question is 
to be deemed and accredited acceptable T Department of Personnel furthe 
reserves the ripht to cancel and annul any question wh ever, upon juiry, it 
cecms that none of the Jisted ‘answers thereto can properly be considered accept- 


able. 


key for this test will be posted on the Official Notices 
plication Section of the Department of Personnel, 
Ny 


rk, N.Y. 10013, for a period of three days beginning 


Please do not telephone the P:partment of Personnel requesting information as to 


the progres 


Such inrornmation will not be 
available to candidates. All candidates in this examination will be notified 
individually by mail of the rating they receive when the rating of all candidate 

in the examination has been completed. Notify this Department promptly in writing 
of any change in your address to insure prompt delivery of such noti 
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eh : pa én an ae Ske +“ Appl. lio. 
rte DEPARTMENT OF PERSONNEL —-+ 
Oe 
NW] 2 Roce PKOMOTION TO HUMAN RESOURCES SPECIALIST 
Nd *. EXAMINATION NO. 1625 
RUN: 24 
i yeh ; Written Test - Veight 85, 70% Required 
a F y / e 
OY: s 
Cr 
Ys 
¥ 
Octoher 14, 1972 GENERAL INSTRUCTIONS Time Allowed: -2'5 Hour 


Question booklets distributed. READ THIS PAGE .ONLY. 
Open your booklet. Begin work. 

End of test. Stop all work. This signal is 

Given 24; hours after the ond gn 

you finish befere the third al, summon 

the mcuitor. No extra time is allowed on 

account of lateness. The time allowed includes 
time for fingerprinting. 


4 
BOOKLETS This test booklet should consist of W@W consecutively nuthered 
pages. There are 80 questions, all ct equal weifht. ANSWER 


1 


ALL QUESTIONS. Examine your booklet after the 


ignal. If 


is piven to be sure it is not defective in any 
responsible for obtaining a complete booklet. 


You may make a record of your answers in the test booklet and 
you may take the test booklet with you. This is necessary in 
the event that you wish to protest the proposed key answers. 
The time allowed includes t'me for recording your answers and 
no time may be taken for this purpose after the third signal 
is given. Further information concerning key answeis and 
protests appears at the end of the test booklet. 


If you read these instructions carefully, you will not need 
to make inquiries. No monitor is authorized to change these 
instructions or the terms and conditions of this test. You 
will be DISQUALIFIED if you obtain or attempt to obtain in- 
formation or help from books or other sources, copy or at- 
tempt to copy or read the work of another candidate, permit 
your work to be copied or read by another candidate, com- 
municate in any way with another candidate, leave your seat 
without permission of the monitor, sien your name or put 

% any identifyinz mark on any answer sheet, or disobey an in- 
q struction of an examiner. You are prohibited from having on 
your desk or in your possession during the course of the 
test any reference materials, notes, or newspapers which 

you have brought with you, 


It is a misdemeanor for anyone to impersonate a candidate 
or permit himself to be impersonated at an examination. 


Smoking is NOT permitted. 


LEAVING During the test you may not leave your seat except when neces- 
sary, and then only when accompanied by a monitor. You may 
Withdraw without completing this test and not have your test 
rated by writing "I withdraw" and your signature on your an- 
swer sheet and giving it to the monitor. ‘'lowever, no candi- 


date will be permitted to depart before 10:30 a.m. 


DO NOT OPEN THIS BOOKLET WNuIL THE SECOND SIGNAL IS GIVEN 
4 
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DIRECTIONS FCS ANSWERI UESTI 

Answer all questions on the official Answer Sheet before the third bell has rune 
ONLY YOUR ANSWER SHEET WILL BI ATTD. 
For each question, select that answer which you believe to be the acceptable one 
among those listed and acte the capital letter preceding your selecte inswer 
Then, on the official answer sheet, find the row numbered the same as th question 
and record your answer by blackening the space between the dotted ling immediately 
to the right of the capital letter representing your selected answer, using a No. 2 


pencil. 

When recording your ansver with a No. 2 pencil, fill in the entire space between the 
dotted lines and the full lengt Ft} 

using several strckes. 


1 Of the lines by pressing firmly on the pencil a 


IF YOU WISH TO CHANGE AN ANSWER, TRASC THE OLD ANSWER COMPLETELY. Do not cross it 


out. “en record your new answer in the proper space. 


Do not make any marks except your ansver. 
improperly erased rs or stray marks ca 
may be comnted as answers. 


an one ansver to a question or 
resting the pencil on the paper 


Use pen (blue or black ink) for filling in the headings on your answer sheet but use 
onlw the No. 2 pencil for recordinp your answers, 


Do not fold the answer sheet or damage the edges because the rating is done by 
machine. 


Since there are only 4 answers listed for euch question, the spaces to the right of 
the letter FE on the answer sheet are not used for this examination. 


To help you understand the procedure, the following sample item is given: 
SAMPLE 9: The sum of 5 and 3 is 
(A) 2 (B) 8 (Cc) 9 >): 21. 


The sum of 5 and 3 is 8, so that the acceptable answer is shown thus on your answer 
sheet: 


SAMPLE 9: A ==s==  — oo c 


CONTINUE ON 


PAGE 


The following are the finel key answers as 


meeting held on 
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EXAMINATION NO. 1625 
PROMOTION TO HUMAN PESOURCES SPECIALIST 
Final Key Answers for Written Test 


Nald October 14, 1572 


consideracion of all protests submittcd by candicatcs. 


21. B 41. C 61. 
22. D 42. A 62. 
23. A 43. D 63. 
24. C 44.4 64. 
25. A 45. D 65. 
26. A 46. C 66. 
27. B 47. .D 67. 
28. B 48. B 68. 
29. A 49. C 69. 
30. B 50. C 70. 
31. B 51. B 71. 
32. D 52. C 72. 
33. A 53. A 73. 
34. C 54. B 7%. 
35. A 55. D 75. 
36. C 56. D 76. 
37. A 57. B 7?. 
38. C 58. A 76. 
39. A 59. C 79 


adopted by the Commission at a 
These key answers result from careful 


Or» 
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The terr "first-line supervisor" refera to the luwest level of surervision in 
organization. A dilemma faced by the first-line supervisor is that he repres t 
(A) manavenent (C) manazenen labor 

BR labor ()) netther ement nor labor, 


Management experts generally consider it advisahle to Rive inetructions orally, 
even thourh these instructions mav later be put in writine for nerrnanent 
reference. The main reason for this advice is that 


(A) employees sometimes misplace written instructions 
(B) explanations can be made in accordance with individ: eeds 
(C) written instructions tend to be unclear and ambiruo > 


(D) employees resent being given instructions in writine 


Of the following, the best reason why a supervisor should not delegace a certain 
job to a subordinate is that 


(A) he does not have any subordinate who can develop the skills needed to do 
the job 

(B) it is easier and quicker to do it himself 

(C) he knows it will be done correctly if he does it himself 

(D) he 2 4oys doing it himself. 


4 
Of the following, the step which the supervisor should take first in handline a 
complaint from a member of his etaff is to 


(A) gather background informaticn relevant to the complaint 

(B) establish tentative solutions or answers to the complaint 

(C) determine the nature of the comolaint as clearly and as fully as possible 
(D) make a determination as to whether the complaint is valid. 


Before a supervisor delegates one of the duties which he normally performs to a ” 
member of his staff, the first thing he should do is 


(A) determine the lonr-range purpose of the Job 
(B) determine exactly what tasks the job involves 
(C) decide how long it takes him to do the job 
(D) decide to whom he will assign the job. 


When an employee is not sure of the intent of a policy statement, it is usually 
best for him to consult 
(A) a fellow worker (C) his supervisor 


(B) a member of the planning staff (D) the manual of procedures. 


The most appropriate tine for a supervisor to have a discussion with an employee 
who has violated an agency policy is 


(A) as soon as possible after the violation has occurred 
(B) after a cooline-off period has elapsed 

(C) the day after the violation occurred 

(D) durinze the next staff meeting. 


10. 


Li. 


33. 


14, 
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(A) infor taff of the latest administrative pwlicies 

(B) obtain the benefits cf collective thinking about a problem 

(C) let staff knoyv that he is aware of violation: of personnel policies 
(D) pive dissat ied employees a chance to voice their grievances, 


Of the followine, a term used to descrihe how a 5s pervisor may determine whether 
he is communicating effectively with his staff is called 


) implementation 
D) delegation. 


False rumors about unpleasant possibilities such as employee cutbacks can do 
serious damaze to morale. Most rumors of this kind in larec organizations and 
public agencies are caused by 


(A) over-pernissiveness and general laxity of supervision 
(B) a breakdown in communication tveen management and employees 
(C) employees who distort the facts for their own purposes 


(D) newspaper articles planted by special interest groups. 


Of the following, staff meetines are least likely to be productive when 
(A) only four or five people are present 
(B) the chairman conducts the meetine i 
(C) discussion is kept to a minimum 

(D) private discussions are not alloted 


n a formal manner 


The one of the following versons who usually would be classified as belonging 
to middle manarement is the 


(A) senior clerk \.) »=«oureau director 
(B) aeency head () deputy commissioner. 


Mf the following, tne best wav for a person to develop comnetence as an interviewer 
is to 


(A) attend lectures on intervievine techniques 

(B) practice with emplovees on the job 

(C) contuct interviews under the sunervision of an experienced instructor 
(D) attend a trainine course in counsellinre. 


Of the followins, the tyne of emplovee sho would probablv expect to be eiven the 
most authority to use indenendent iudpment is the 


(A) chemical encineer (C) hookkeener 
(B) clerical worker (D) reristered nurse. 


Assume that you are asked to study and report on employee turnover in several 
azency units which vary widely as to total number of emplovees and the number of 
employces involved in turnover, In order to present an accurate picture of 


turnever, vour renort should show, with rerard to persons leavinr, both actual 
Mumbers and 


(A) central tendencies (Cc) aw data 
(B) percentages (D) rounded totals. 
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Manv emo! es tend to resist a reorpanization because hey feel that thety 
Status and s srity are threatened Of the fol} Ing, the } t te 
casier for e~nloyees to accept the chances wecessitat Vv reorpanuization i 


(A). introduce many chances at the same time 
(B) fmive them a chance to Participate in cvaluat 
(C) keep the chances secret until they are put into effect 


(D) have staff people who h had little contact wit e affected 
initiate the chanres. 

The most important reason for investigating every accident on the job is tc 

(A) find out who was resnonsible for the accident 

(B) determine the Organization's Jeral liability for the accident 

(C) correct the conditions or actions which caused the ; ideat 


(D) discipvline the emplovee who caused the aecident. 


Research studies indicate that an important difference 4 tween hizt-productior 
and low-nroduction supervisors lies in their manner of hen4linpe mistakes. 


subordinates make mistakes the hieh-production supervisor probably would 
’ 


(A) concentrate on fixing responsibility and determining the subordinate’: 
excuse for the mistake 

(B) take over the assirnment himself in order to avoid recurrence of the 
mistake 

(C) look upon the mistake as an opportunity to provide training 

(D) give the assignment to a subordinate who is not likely to repeat the 
mistake, 


The use of statistical controls is generally considered to be one of manareren 
most effective means cf determining what is happening at the operating level 
an agency. Of the following, statistical controls are least useful for 


(A) furthering coordination (C) setting standards 
(B) measuring rorale (D) pinpointing responsibility. 


A basic probleri of the supervisor is how to motivate employees. One approach 
to “ioternalize" Motivation by providine opportunities for ennlovees to derive 
satisfaction from the work itself. Of the following, internalized *.Otivation 
would be the least effective approach where the emp loyee 


(A) enjoys autonomy because of the nature of the job 
(B) accepts the organization's obicctives 

(C) makes the job his central life focus 

(D) does a routine or assembly line joh. 


Of the following, an important goal of the current reorganization of the New 
York City Human Resources Administration is to 


(A) strengthen the centraliz ion of servi: at arency headquarters 

(B) provide services within the neiehberhoods, according to local needs 

(C) equalize the distribution of responsibilities between headquarters 
and neichborhood of fices 

(D) eive more authority and responsibility to neichborhood offices than 
to headquarters. 
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The ¢ t fel) 1) which is no rpurnose of the movement toward cecentral- 
izatic f ¢ Yo Cit Over? t 4 t 

(A) reduce citize tlienatic 

(%) holster Cit rvices 

(C) 1 9 j 1] eC€ 

(Dp) di x the local power structure. 


Of the following, the most desirable wav to strenpthen the capacity of cormnunities 


solution of their own problems is to 


(A) encourare participation of local residents in service nlannine and 
delivery 
(B) estehlish city-wide job training prorrams 


(Cc) -e technical assistance to local small business so that thev will 
by experience 
(D) local residents accountable to government acencies for funds 


and services provided. 


} 


In organizing the residents of a disadvantazed neighborhood to develop projects 
for community improve t, the most effective of the following approaches is 
to 


(A) concentrate on the group with most members 

(B) devote more attention to prouns which have est d interests 
(C) trv to include all groups 

(D) give special consideration to official group: 


attempts are made to obtain conmunityv narticipation in a project such as a 
neichborhood improvement program is 


(A) domination by an aggressive but unrepresentative proup 

(B) public oppositien by representatives of government agencies 
(C) frapmentation and disruption of community services 

(DP) serious deterioration in the quality of services. 


Of the followine, the most important aim of the community organizer in his 2arlv 
contacts with a community group should be to 


(A) build a core of common interests 
(8B) establish himself as a forceful leader who can make decisions 
(C) inform the proun of its lepal rishts 


(D) curb discussion of opposing viewpoints in order to develop harmonious 
relations. 


The one of the following which is the best method of encouraging neighborhood 
people to attend a community meetine is to 


(A) send out notices at least a week in advance 

(B) set up an anenda that deals with issues of serious local cencern 

(C) invite a preninent public fieure to address the meeting 

(9) send invitations to community people with similar viewpoints on the 
problem to he discussed. 
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A basic difference between pure experimental research 
experimental research is pvrinarily concerned with ti 


scientific or technolorical ; neralization, while action 


research is 


(A) based on the results of trial 
B) designed to effect improvement in an on-going process 

(C) intuitive rather than scientific in nature 

(D) primarily concerned with the analysis of data for universal reneralization. 


error 


Assume that a community worker is assigned to orpanize a client Proup to 


I 


participate in planning for services thev particularly need. Of the following, 


it would he least immortant for the organizer to become familiar with the 
(A) local store owners (C) potential leaders 
(B) neighborhood resources (D) informal leeders. 


Of the followinz, the factor which is most important in encouraging a high level 
of local participation in community projects is the 


(A) degree of sophistication of the local people 

(B) attitudes of the community deve’opnent workers towards the local 
people 

(C) amount of money availadle for trainine 

(D) amount of time available for stimulation of community interest. 


Assume that you are the discussion leader of a meeting of a group of residents of 
4 poverty area, many of whom are apainst a Droposal to locate a methadone 
maintenance treatment center in the nei¢e ‘orhood. The best way’ for you to assist 
the group is to help them first to 
(A) get to know each other on an informal basis 
(B) understand the overall] backeround of the drug vroblem and the need for 

such a facility in the connunity 
(C) concentrate on all the issues until thev iron out conflicting viewpoints 


(D) discuss the pros and cons briefly, take a vote, and accept the decision 
of the majority. 


Of the followine, the most important aim of the community develcpment process is 
primarily to strenethen the 


(A) lone-established socia! a-- political pattern of relationships 
(B) influence of dominant ethnic and religious groups 

(C) lons-standine power of traditional central government 

(D) positive inpulses of people workin” toward a common poal. 


One method of influencine human hehavior is hased upon an optimistic belief in 
human potential for development and betterment. Workers in the human services 
who apply this method expect that it will 

(A) awaken initiative in clients 

(B) demand a pre-chosen response from clients 

(C) pive clients a spirit of competition 

(D) pet clients to accept new ideas. 


UI 
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In the nion of experts, the cosununity development precess is most effective 
when 

(A) final decisions are made solely by the community development exnediters 
(B) a firm plan is wade after : project ects under way 

(Cc) de sions are loft in the hands of couwmunity peopl 


(D) i 


tt factions are permitted to take a firm stand. 
Of the following, the most sirnificant indication that an organizer of a 
community grour has done an effective job ild be a situation where the proup 


(A) continues to prow in size and strength after the oreanizer has departed 
(B) rses after the oreanizer has departed becaure it has fulfilled its 


(C) 


(D) has a warm and 


The main purpose of sroup discussion of community issues by local residents is 
to 


(A) present a predetermined point of view 

(B) provide an outlet for release of the Pparticipants' agrressions 
(C) consider and work through connon problems 

(D) improve relationships among participants. 


37 Of the following, the most important reason why the community development worker 
in a disad\ antaved community of a major metropolis should seek to understand the 
motivations of the local residents is that he will be more capable of assi: tine 
them in developing 


(A) self-help activities 

(2B) projects which will get publicity 

(C) an overall nuster plan 

(D) piojects which do not require technical assistance. 


38. At certain times, there is a tendency for community groups to disregard democratic 
procedures in making decisions, particularly in a situation where 


(A) action depends on availablity of community services 
(B) the executive board makes the decision 

(C) there is need for speedy action 

(D) there is no sound basis for the decision. 


aos The employment of residents of poverty areas with little or no oducational 
P y 
qualifications to assist professional staff members in working with clients of 
human services agencies is generally considered 


(A) advisable, mainly because local paraprofessionals can be expected to bridge 
the gap hetween the middle class professional worker and lower class 
recipients of service 

(B) inadvisable, mainly because the employment of workers who are not professionals 
will lower the professional standards of the agency's staff 

(C) advisable, mainly because employment of paraprofessional local residents 
will save agency funds 

(D) inadvisable, mainly because clients will receive services of poorer 
quality than services provided by professional workers. 


40. The social work activist who was a leader in the movement to achieve welfare 
reform by orpanizing welfare clients and encouragine the poor to demand their 
lezal rights to puplic assistance is 


(A) Saul Alinsky (C) Sertram Beck 
(B) Richard Cloward (D) Jcsse Gray. 
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Ansver questions 4! through 46 solely on the basis of the followtup passare. 
Too often in the past secietly has accepted the existinr social welfare proprame, 
preferrinp to tinker with refinements when fundamental reform was in order. It has 


degrading welfare system in which the instruz 
and inently used. Jt 


those forced to benefit from it demoralized those who had to administer it at 
the level where the pain was cleariy visible. 


nt of government 


has been a system which hag only alienated 


There is a need to put this nation on a course in which cash Lenefits, providing a 
basic level of support, are conferred in such a way as to intrude as little as 
possible into privacy and self-respect. It is difficult to define a basic level of 
support, no matter how high or low it might be set. In the end, however, the 
decision is not determined so truct 


by how much is truly adequate for a family to 
meet all of its needs, but by the resources available to carry out the promise. 
That may be a harsh fact of life but it is also just that - a fact of life, 


b> 


Of the following, the most suitable title for the foregoing passage would be 


) The Need for Government Control of Welfare 
1) Determining the Kasic Level of Support 
) 

) 


The Need for Welfare Reform 
The Elimination of Welfare Proerams. 


In this passage, the author's greatest criticism of the welfare system is that it 


(A) disrespectful of recipients 
(2) expensive to administer 
t 
(C) limited by regulations 
(D) widespread in application. 


According to the passage, the basic level of support is actually determined by 


(A) how much is required for a family to meet all of its needs 
(Bb) the ace of the recipients 

(C) how difficult it is to administer the program 

(D) the economic resources of the nation. 


In this passage, the author docs not argue for 


(A) a work incentive system 
(B) a basic level of support 
(C) cash benefits 

(D) the privacy of recipients. 


As used in the foregoing passage, the underlined word demeaning means, most nearly, 


(A) ineffective (B) expensive (C) overburdened (D) humiliating. 


As used in the foregoing passage, the underlined word ineptly means, most nearly, 


(A) foolishly (B) unsuccessfully (Cc) unskillfully (D) unhappily. 
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Answer questions 47 threuch 50 solely on the basis of the following naraeraph. 
The unerployvment rate, it those unemmloyed in the sense that they ar 
actively lo ing for "or Re Ue find it, give 1 relative] t f 5.¢ ] 
index ot economic conditio community. <A bette index is the le nt 
rate which includes the unewployment rate and also includes those working part-tire 
while they are trying to set full-time work: those heads of househol: ler 
years of aze who earn less than $65 ver week working full-tise, enc those ind } 
under 65 uho are not | is of housel and evn less than 6 per veel na full- 
time tov: and an estimate of the males ‘not ceunted," VOX eal con 
in phetto areas, 

47, %f the following, the most suitable title for the forcpoinre paragrayh would he 


) Employment in the United States 
>) Part-time Workers and the Feonony 
) The Labor “farket and the Comminity 
) Two Indicators of Feonomie Conditions. 


48. On the basis of the paragraph, which of the following statenents is correct? 


(A) The unemployment rate includes evervone who is not i llv employed, 
The subemployrent rate is higher than the unerployment rate. 
The unerployment rate vives a more complete picture of the economic 
situation than the subemplovyment rate. 

(D) The subemployment rate indicates how manv part-tire workers ar 
’ dissatisfied with the number of hours they work per week. 


| 49. As used in the foregoinr paragraph, the underlined word superficial means most 
rearlyv 


| (A) exareerated (B) official (C) surface (D) current. 


50. Accordine to the paragraph, which of the following is included in the suhennloynent 
rate? 


(A) Evervone who is unemploved. 

(B) All part-time workers. 

(C) Fvervone under 635 who earns Jess than $56 per week in a full-time job. 

(D) All heads of houseliolds who earn less than $60 per week in a full-time joh. 
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A. Food 


Shelter 
C. Clothine 
} 


Nousehold Operation 


FE. “Medical Care 


7 


Transvortation 


G. Miscellaneous 


Accordine to the diagram, the percentage spent on medical care is 


Fok Fame (B) 8% (C) 162 (PD) 18% 


According to the diapram, the total percentage spent on food, shelter, and 
clothing is 


(A) 55% (B) 60% (c) 707 (D) 75% 


2 


In a typical weriod, if the family spent $30 on transportation how much did it 
spent on household operation? 


(A) $120 (B) $192 (C) $399 (Dp) $480 

If the family income is $125 a week, how .uch does it spend on transportation each 
year? 

(A) $60 (B) $2606 (C) $600 (D) $2,600 


nur 
e ; ~ 6 £nNN severa years. LASSE 
Assume that the annual income of a fanilvy was >,49% for seve : 1 i i nya 
ing ' mic er rc are ¢ 
the incore went un 5”, and the family then trfnled the tvnical percentar — 

! iten ‘ear \'as most nearly 
household operation. The areunt svent on this item last vear \ 


(A) $291 (F) $1,134 (Cc) $1,296 (D) $2,721 


56. 
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Answer questions 56 throurzh 69 solely on the basi: of the table presented below 


AFDC Farily “embers in Urbaneille 


keferred to and Fnrolled in WIN Prearan, 1969-1571 
Referred 
197] 
Mother 1,306 
Father 743 950 
Child, age 170 222 
16 and over 


According to the table, how many AFDC family members were referred to "IN in 1971? 


(A) 1,792 (B) 2,004 (C) 2,388 (D) 2,478 


Accordine to the table, the number of AFDC children, 16 and over 


r, who were enrolled 
in ‘IN in 1971, was greater than the number enrolled in 1969 by 


(A) 12 (B) 34 (C) 38 (D) 52 


According to the table, the number of AFDC mothers who were enrolled in 197] 
increased over the number enrolled in 1969 most nearly hy 


(A) 20° (B) 32% (C) 54% (D) 83 


In 1972, if the number of AFDC mothers referred to VIN increases 5% over 1971, the 
number of fathers referred increases 8% over 1971, and the nurber of children 
referred increases 5% over 1971, the total number of AFDC family members that will 


be referred in 1972 is most nearly 

(A) 1424 (B) 1524 (C) 3139 ‘ (D) 3990 
According to the table, the Percentase of AFDC fathers not enrolled in \V'IN in 1971 
of the number referred that year is most nearly 


(A) 23% (B) 25% C6). FLEE CD). 77% 
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f t f51] I ( cou for the lar t portion of tie uiget of 
the | ‘ ir¢ 1 t ' n is 
(A) personnel and rt ser 
() r lie t e and edicaid 
( ) er t r i 
(D) « ty sation a evelopnent 
Accord to the ] t statist publi by the | Jenartrment of lealth, 
Edu j el See tate whicl spenk thn lar t amount of monev per person 
for } lic istar i 
(A) California (2) Massachusetts (C) Pennsylvania (D) New York. 
Accordi to the t recent UV. Census “jureau Report, the group living in 
lew Yor t iict as the lovest income level is tt 
(A) lac (1.) Puerto Ricans (C) Dominicans (D) Haitians. 
The er . that contains the largest number of individuals receivine public assistance 
is 
(A) children under working age (C) the aged, disabled and blind 
(B) unew>loyved heads of families (D) unemployed single persons. 
A major difficulty faced by new arrivals to cities since 1959, which did not exist 
for earlicr Furopean immirrants, is the fact that the majority of present-day 


arriv 


(A) 
(B 

(<) 
() 


7 
i 


must forfeit their native culture patterns 

hac — | , ? L kit 1 1 ene 7 ina a tia » 
have an obviously srker skin color t!ian most Jong-time residents 
have little education 

Lave few occunational skills. 


Generally sneakine, low-income persons io not make maximum use of opportunities and 
services available to them, mainly becauss 


most naraprofessional workers, while sincere in the desire to serve, 
are unable to reach the thiard core 

much of che routine paperwork in pul lic assistance progiams is now 
assiened to paraprofessional workers 

thev have become increasingly self-reliant and prefer to cope with 
their problers withour. help 

they lack the confidence and know-how necessary to make their needs 
known to the prerer rersons or azencies. 


The one of the following problems which has most recently become ‘a serious concern 
of the Youth Services Age v is the 


(A) 
(*) 
(C) 
(D) 


increasing hieii school drop-out rate 
resurvernce of firhtine voutl ganrs 
t 


spread of yout’: narcotics addiction 


Jack of recreation programs. 


Nf tie follovine, the most recent Jevelopnrent with reeard to welfare recipients is 


(A) 


(2) 
(C) 
(0) 


introduction of the declaration of need instead of an investication of 
elinibility 

a najor enphnasis on emnloyment progzrans 

increased use of casework therany and psychiatric counselling 
acceptance of narcotics addicts for home relief. 
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¢ nt of the i ree sii crest t s olicy for the 

I t t » coo! ic d aliocation of ft for cc lity acttun 
es the 
i : 

( inity Develor t Asency 

l rtment of Social ‘ »S 

( 1} on Povert 

M er and Career Development Age 
policies of the Council Against Poverty are carried out by the 


velopment Agency 


cial Seivices 


Manpower Service Centers. 


reorganization 
Ors 


current 


unit of 


of the Human Resources Administration has established 


inization for provisions of services at the neighborhood level as 


ignated poverty area 
district 


(C) catchment area 


resources (D) census tract. 


Child Development Commission recently establish di by the Agency for Child 


Development can best be described as a group comprised of 


(A) 
(B) 


(Cc) 
(D) 


The 


of Social 


professionals in child psychology and early childhood education who 
will consult with Agency staff members on policy and programs 
parents, community organizations and concerned citizens, who will 
help the Agency determine, review, and modify policies and guide- 
lines for child-care services 

child-care experts who will provide technical assistance to private 
mroups that want to develop early chilchood centers 

professionals who will offer health and nutrition consultation, and a 
variety of support and referral services for children and parents. 


basic purpose of the new office of Community Social Services in the Department 
Services is to 

help local community leaders establish liaison with rivate 
service agencies their communities 

determine the social service needs of each comunity and provide 
services in accordance with these needs 


soctal 
in 


provide information and referral to ali HRA services existing in a 
particular community and to services provided by other City agencies 
and private organizations 

assume responsibility for a varicty of social services mandated by 
regulations. 


Federal and State 
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The Social Service ixcheupe is correctly de rjibed as 

(A) recruftuent center for the training aud placement of voluntecrs for social 
and health ayenet 

(2B) cente:x ich utains ac ords of families ard 
indi juals known to : i 

(C) center ich provides infc tion abo referrals to social 
and health agencies aid vroprietary pursin 

(D) confidential advisory service to help potential contributors evaluate 


local voluntary health and welfare agencies. 


Which one of the following is an imnortant purpose of the formation of the Of “ice 
of Special Services for Children in the Department of Social Services? 


(A) Greater proerammatic integration of the protective and supportive services 
to childien who are abused, neslected, dependent, delinquent or in need 
of services. 

(B) More professional attention to child abuse cases and prompt ccurt action 
to penalize parents of abused or neglected children, 

(C) Separation of prorrams and fuecilities for children adjudged io be 
delinquent from special services for other dependent, abused or neglected 
children. 

(D) Increased attention to home-findine and foster care and adoption 


services rather than institutional care for dependent children. 


The one of the following which is provided by the Department of Social Servicc;s 
for current, former and potential public assistance recipients only is 


(A) information services (C) referral services 
(B) child welfare cervices (D) homemaker services. 


A major moal of the Department of Social Services which is part of the current 
reorganization and the separation of income maintenance from social services is 
to 


(A) limit the provision of public social services to those New Yorkers 
who are eligible for public assistance 

(B) make public social services available to all New Yorkecs, whether or 
mot they require financial assistance 

(C) refer clients who require social services to private avencies wherever 
possible 

(D) emphasize casework treatment and refercal of clients for fsychiatric 
services rather than programs to effect environmental change. 


Of the following, the main functions of the Manpower and Career Development Agency 
(MCDA) of the Human Resources Administration ere to 


(A) train the unskilled, unprade existine skills, develop job opportunities, 
and place newly-trained people in jobs 

(B) operate mannower, recruitment and testine ce-tors under contract with 
private oreanizations 

(C) provide remedial education and follow-up for disadvantaced potential 
collerne students and vocational testine and counsellin for veterans 
and ex addicts 

(D) provide job development, interviewine and Placement, and manpower 
research services. 
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h. S - COLLECT! You t - the que 1 bool t with you hen you 

heve conp] t t re {r eat ‘ ! itor. Tise nitor will 

pi up y & ct. Leave the buildi quickly quiectl You should 

mal f your ication n er and include thi ber in any future 

sorre ( ith t cpartment Personne] with respeet to this examination. 

KEY ANS ‘st PROTEST The propoced key a ers to this test will he posted 
the Off l Notic 4ulletin Board of the Application Section of t 

De tnent of Personnel, 49 Thomas Street, New York, N.Y¥., beginning 10/25/72 

for a period ef three days. Candidates who wish to file protests apainst the 

pr sed k nswers must submit in writing such protests tozether with their 

reasons for ich protests. Such protests must be postmark2¢d not later than 

11/15/72, and must contain the candidate's application number. Protests 

should be addressed to t Jepartmect of Personnel, 55 Thomas Street, New York 

N.Y. 19013, with the examination title written on the face of the envelope. 

Protests postmarked later than 11/15/72 will not be consideied. 

Protests sho. id be clear and legible, preferably typewritten. Fach protest ust 

begin on a parate pace and state the number of the question being: protested. 

Protests siould berin ty indicating which ansver the candidate considers correct, 
1 sl esent r ied arguments and clearly identify any quoted sources 


c 
s. Each pare must be signed by the candidate and state his address. 


c 
submitted by individual candidctes will be considered. 


After the Department of Personnel has considered all protests, properly filed, it 
will make a final determination thereon and will thereafter post the final key on 


the Official Notices Bulletin Board of the Application Section of the Department 


of Personnel for a period of three days. Such rosting constitutes official public 
notice by the Department of Personnel to all cracidates of the final determination 


in this connection. 


The Nepartrent of Personnel, in establishing “ey answers to this test, reserves 


the right to determine which of the answers listed for each question is to be 
deemed and accredited as acceptable, ard whether mere than one of the answers 
listed for each question is to be deemed acceptable and accredited as such. The 


Department further reserves the right to cancel and annul any question whenever, 
upon inquiry, it deems that none of the listed answers thereto can properly be 
considered acceptable. 


Please do not telephone the Departre 


yt tel’ e the D nt of Personnel requesting intocmation as to the 
progress of the ratin2 of this examination. Such information will not be available 
to candidates. All candidates in this examination will be norified individually 

by mail the rating they receive when the rating of all candidates in the 
examination has been completed. “Notify this department promptly in writing of any 


change in your address to insure pronpt delivery of such notice. 
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The term "first-line supervisor" referq to the Lowest level of supervision . 
organization. A dilewma faced by th firsi-}ine supervisor is that he represents 
(A) management (C) manarement and labor 


(B) labor (D) neither management nor lebor. 


Manarement experts generally consider it advisable to give instrvctions orally, 
even though these instructions may later be put in writiny for permanent 
reference. The iain reason for this advice is that 


(A) employees sometimes misplece written instructions 

(B) explanations cen be made in accordance with individual needs 
(C) written instructions tend to be unclear and amhisuous 

(D) employees resent being given irstructions in writine. 


Of the following, the best reason why a supervisor should not delegate a certain 
job to a subordinate is that 


(A) he docs not have any subordinate who can develop the skills needed to do 
the job : 

(B) it is easier and quicker to do it hinself 

(C) he knows it will be done correctly if he does it himself 

(D) he enjoys doing it himself. 


Of the following, the step which the supervisor should take first in handlinz a 
complaint from a member of his staff is to 


(A) gather background information relevant to the complaint 

(B) establish tentative solutions or answers to the complaint 

(C) determine the nature of the complaint as clearly and as fully as possible 
(D) make a determination as to whether the complaint is valid. 


Before a supervisor delegates one of the duties which he normally performs to a 
member of his staff, the first thing he should do is y 


(A) determine the long-range purpose of the job 
(B) determine exactly what tasks the job involves 
(C) decide how long it takes him to do the job 
(D) decide to whor he will assign the job. 


When an employee is not sure of the intent of a policy statement, it is usually 
best for him to consult 


(A) a fellow worker (C) his supervisor 
(B) a member of the planning staff (D) the manual of procedures. 


The most appropriate time for a supervisor to have a discussion with an employee 
who has violated an agency policy is 


(A) as soon as possible after the violation has occurred 
(B) after a coolins-off period has elapsed 

(C) the day after the violation occurred 

(D) during the next staff meeting; 
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(D) newspaper articles planted by special interest groups. 
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of the following persons who usually w 
le management is the 


nior clerk (C) hure 


ely to be productive when 


rmal manner 


ould be classified 


au director 


ency head (9) deputy commissioner. 


followinaz, the best wav for a nerson to develop comnetence 


tend lectures on intervievine techniaue 
actice with emplovees on the job 

nduct intervi-ys under the sunervision 
tend a trainine course in counselline 
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that vou are asked to .wtudv and report 
units which varv widely as to tota) nur 
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anv emplovees tend to resist a reorganization because they feel tna their 
Status and security are threatened. Of che following, the best wavy to make it 
easier for employees to accevt the chanres necessitated + reors tion is to 
(A) introdu many chanees at the sane time 

(B) give ther a chance tn particivate in evaluating nronosed changes 

(C) keep the changes secret urtil thev are put into effect 

(D have staff people who have had little canta t with the affected ervloyee: 


The most important reason for investigating every 


(A) 
(B) 
(C) 
(D) 


initiate the chanres. 


tident on the job is to 
find out who was responsible for the accident 
determine the oreanization's legal liability 
correct the conditions or actions which caused the accident 
discipline the employee who caused the accident. 


for the accident 


Research studies indicate that an important difference betyeen hizh-production 


and low-production supervisors lies in their manner of ina: 


iline mistakes. When 


al 


subordinates make mistakes, the hich-production supervisor probably would 


(A) 
(B) 


(C) 
(D) 


The use of statistical controls is eenerally considered to be one cf manarement's 


concentrate on fixing responsibility and determining the subordinate's 
excuse for the mistake 

take over the assignment himself in order to avoid recurrence of the 
mistake 

look upon ihe mistake as an opportunity to provide training 

Rive the assignment to a subordinate who is not likelv to repeat the 
mi* take. 


most effective means of determining what is happening at the operating level of 


an agency. Of the following, 


(A) 
(B) 


A basic problem of the supervisor is how to motivate employees. 


Statistical controls are least yuseful for 


furthering coordination (Cc) 
measuring morale (D) 


setting standards 
pinpointing responsibility. 


One approach is 


to “internalize” motivation by providing opportunities for employees to derive 


Satisfaction from the work itself. 


Of the following, internalized motivation 


would be the least effective epproach where the emplovee 


(A) 
(B) 
(Cc) 
(D) 


enjoys autonomy because of the nature of the job 
accepts the organization's objectives 

makes the jo) his central life focus 

does a routine or assembly line joh. 


Of the following, an important goal of the current reorganization of the New 
York City Human Resources Administration is to 


(A) 
(B) 
(Cc) 


(D) 


strengthen the centralization of services at agency headquarters 
provide services within the neighborhoods, according to local needs 
equalize the distribution of responsibilities between headquarters 
and neighborhood offices 

give more authority and responsibility to neichborhood offices than 
to headquarters. 


24. 
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27. 


276a 
Exhibit N-4 Annexed to Defendants’ Stipulated Facts 


he one of the followine wich i not 2 nurp e of the moverert toward decentral- 
igstion of tlew York City r tis! 

(A) ¢ cl 1 alie tic 

(B) ) r Cir ic 

(c) } 2 + len } ’ 

(D) t t loc pover structur 

Of ti f ovine, the ost desivable 1 to strenrt the canacity of communities 


to contribute to the solution of their own problems is to 


irticipation of local residents in service nlannin> and 
itv-wide job training procrans 


nical assistance to lecal small business so that they will 


,idents accountable to government agencies for funds 


residents of 


mprovement, the 


aged neighborhood to develop projects 
ive of the folloving approaches is 


(4) concentrate on the group with most 
(B) devote more attention to groups 
(C) try to inclu all groups 

(D) give special « {deration to official erouns. 


embers 


h have vested interests 


The one of the following which has been the most common vrohlJem which occurs when 
attempts are made to obtain community participation in a project such as a 
neichborhood improvement profram is 


(A) domination by an aggressive but unrepresentative group 
(B) public opposition by representatives of government agencies 
{C) fraementation and disruption of community services 


(D) serious deterioration in the quality of services. 


Qf the followings. the most important aim of the community oreanizer in his early 
contacts with a communitv group should be to 


(A) build a core of common interests 

(8) establish himself as a forceful leader who can make decisions 
(C) inform the provp of its legai rights 

(D) curb discussion of opposine viewpoints in order to develop harmonious 


relations. 


The one of the following which is the best method ncouraging neighborhood 
people to attend a community meeting is to : 


(A) send out notices at least a weel. in advance . 

(B) set up an agenda that deals wite issues of serious local concern 

(C) invite a prominent public fievce to address the meetine 

(D) send invitations to community people with similir viewpoints on the 
problem to he discussed. 
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A basic differen between pure exn 


i tin t research and , s esearch is, th 
experdwental rch is prinarily concerned wit 1 aivsis of data for 
scientific or techiunlontical hevaltzation, while «cidon research is 
(A): tased on the r lts of trial and error 
(B) desizned to effect inprovesent in en on-going prcecess 
(C) intuitive rather than ecienttfie in norure 
(D) primarily concerned with t! anaulys of data for uni ). gre cali 
Assume that ke is assigned to ormanize a client eroup to 
participete lees they particularly necd 7 the follc , 
it would for the organizer to become familiar with the 
(A) local store ommers (C) potential leaders 
(B) neighborhood resources (D) informal leaders. 

Of the following, the factor which is most important in encouraging a hieh level 
of local participation in com unity projects is the 
(A) degree of sophistication of the local people 
(B) attitudes of the conrnunity development workers towards the local 
people 
(C) amount of money available for trainine 
(D) amount of time available for stimulation of commur interest, 
Assume that you are the discussion leader of a meeti..rc of a groun of residents 
a poverty area, manv of whom are against a proposal to locate a methadone 
maintenance treatment center in the neighborhood. The best way for you to assis 
the group is to help them first to 
(A) get to know cach other on an informal basis 
(B) understand the overall backrround of the drug problem and the need for 
such a facility in the community 
(C) concentrate on all the issues until thev iron out conflictine viewpoints 
(D) discuss the pros and cons briefly, take a vote, and accept the decision 
of the majority. 
Of the followine, the most important aim of the community development process is 


primarily to strengthen the 


(A) lone-established social and political pattern of relationships 
(B) influence of dominant ethnic and relicious grouns 

(C) lone-standine pever of traditional central eovernment 

(D) positive inpulses of peonle workin? toward a common poal. 


One method of influencing human behavior is based upon an optimistic belief in 
human potential for develcpment and betterment. Workers in the human services 
who apply this method expect that it will 


(A) awaken initiative in clients 

(8) demand a pre-chosen response from clients 
(C) give clients a spirit of competition 

(D) pet clients to accept new ideas. 
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Of the fol! inz, the : t sicenificant indication that an organizer of a 
c it I has dor tive job would he a situation where the group 
(A) continuss to prow in size and strength after the orzanizer has departed 
(B) disperses after the orsanizer has departed because it has fulfilled its 
purposs 
(C) attains its goals only under the organizer's guidance 
(D) has a warm and friendly relationship with the organizer. 


The main purpose of group discussion of community issues by local residents is 


(A) present a predetermined point of view 
(B) provide an outlet for lease of the participants’ agrressions 


(C) considex and work throuzh comnon problems 
(D) ’rprove relationships amone participants. 


Of the following, the most important reason why the community development worker 
in a disadvantaged conmunity of a major metropolis should seek to understand the 
motivations of the local residents is that he will be more ccpable of assisting 
them in developing 


(A) self-help activities 

(R) projects which will get publicity 

(C) an ove ~all master plan 

(D) projects which do not require technical assistance. 


At certain times, there is a tendency for comminity groups to disregard democratic 
procedures in making decisions, particularly in a situation where 


(A) action depends on availability of community services 
(B) the executive board makes the decision 

(C) there is need for speedy action 

(D) there is no sound basis for the decision. 


The employment of residents of poverty areas with little or no educational 
qualifications to assist professional staff members in working with clients of 
human services agencies is generally considered 


(A) advisable, mainly because local paraprofessionals can be expected to bridge 
the gap between the middle class professional worker and lower class 
recipients of service 
(B) inadvisable, mainly because the employment of workers who are not professionals 
will lower the professional standards of the agency's starf 
(C) advisable, mainly because employment of paraprofessional local residents 
will save asency funds 
(D) inadviseble, mainly because clients will receive services of poorer 
quality than services provided by professional workers. 


The social work activist who was a leader in the movement to achieve welfare 
reform by organizing welfare clients and encourocing the poor to demand their 
lesal rights to public assistance is 


(A) Saul Alinsky (C) Bertram Beck 
(8) . Richard Cloward (D) Jesse Gray. 
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I i" , T t . 4 ; ire 
i ft pa t t t f ti } lf roprans, 
! 1 } \ the tr nt a ner 
ne , weed t } } t has onl] lienated 

t f f it an ral ( 2 wh 1 to administer it a 
the level wir the i clearly visible 

There J to put ¢ natic 1 cour in whi ca benefits, providines 
basic level f support, are mnferred in such a vay as to intrude as little a 
P ible into privacy I lf-re ct. It is difficult to define a basic level 
Support, 1 -atter how hi or low it mieht he set. In the end, however, the 
decision is not determined so much by how much is truly adequate for a family to 
meet ail of it eds, but by the resources avatlable to carry out the promise. 
That may be a harsh fact of life but it is also just that - a fact of life. 

f the foll , the 5st suitable title for the foreroinz passage would be 

(A) The Need for Government Control of “elfaré 

(Bb) Teterninine the Kasic Level of Support 

(C) The Need for Velfare Reform 

(D) The Elinination of ''clfare lrozrams. 


In this passaje, the autuor’s greatest criticisn of the welfare system is that it 
is too 


(A) disresnectful of recipients 
(£L) expensive to administer 

(C) limited by repulatioas 

(D) widespread in application. 


According to the passage, the basic level of support is actually determined by 
A) how much is required for a family to meet all of its nee 
) the ace of the recipients 

C) how difficult it is to administer the program 

) the economic resources of the nation. 


In this passage, the author does not areue for 


(A) a work incentive system 
(BP) a basic level of support 
(C) cash benefits 

(D) the privacy of recipients. 


As used in the foregoing passage, the underlined word demeaning means, most nearly, 


(A) ineffective (B) expensive (C) overburdened (D) humiliatine. 


As us2d in the foregoing passage, tie underlined word ineptly means, most nearly, 


(A) foolishly (fF) unsuccessfully (c) unskillfully (D) unhappily. 
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Angevwer questio:; 47 throuch 50 solely on the vces!: of tt fol ti trarcraph, 
The unemployrent rate, which counts t!} ine leyod in the sar t t they are 
actively lool ine for wor!’ and unable to find it, «fyves a relativels inerfics 
index of economic conditior in a conunity. A better index is ¢t : la:mont 
rate which includes the wu: nloyrent rate and al inelud those worsinyg part -ti 
while they are t ir to t uli-ti ork: | Is of he ehol under 
years of are who earn le than $f per week workine full-tire, and the indivi ] 
under 65 who are not hea of households and ei less than $56 per «ce in a full- 
ti job 1 an estimate of the male ‘not counted," wit i 4 very real concer 
in phetto area 
47, %f the following, the most suitable title for the foresoinr parayvraph would be 
(A) Enplovment in the United States 
(RB) Part-time Workers and the Fconony 
(C) The Labor “farket and the Community 
(D) Two Indicators of Feonomic Conditions. 
48. On the basis of the paraeraph, which of the following statements is correct? 
(A) The unemployment rate includes everyone who is not fullv employed. 
(B) The subenployment rate is hipner than the unemployment rate. 
(C) The unemployment rate gives a more complete picture of the economic 
situation than the subemployment rate. 
(D) The subemployment rate indicates how many part-tire workers are 
dissatisfied with the number of hcurs they work per week. 
49. As used in the foregoing paragraph, the underlined word superficial means most 
nearly ne P 


(A) exaregerated (B) official (C) surface (D) current. 


50. According to the paragraph, which of th 
rate? 


e following is included in the subenployment 


(A) Everyone who is unemploved. 


(B) All part-time workers. 
(C) Everyone under 65 who earns less than *°56 per week in a f:1l-time job. 
| (D) All heads of households who earn less ain $60 per week in a full-time joh. 
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f e questi throus! Telvy on t! basd the diapr resent belo 


A. Feed 
B. Shelter 
C. Clothin; 
D. Housebold Operation 
E. “Medical Care 
ws F, Transportation 
G. Miscellaneous 
= gts 


According to the diagram, the percentage spent on medical care is 


(A) 7% (BR) 8° (C) 16? (D) 18%. 


According to the diagram, the total percentage spent on food, shelter, and 
clothing is 


(A) 552 (B) 602% (C) 70% (D) 75% 


In a typical period, if the family spent $30 on transnortation how much did it 
spent on houschold operation? 


(A) $120 (B) $192 (Cc) $390 (D) $480 


If the family income is $125 a week, how much does it spend on transportation each 
year? 


(A) $60 (B) $260 (C) $600 (D) $2,600 


Assume that the annual income of a family was $5,499 for several vears. Last year, 
, 5”. and the family then trinled the typical percentage for 


the income went un 
{tem last year was most nearly 


. - } 
household operation. The amount speat on this 


(A) $891 (PB) $1,134 (C) $1,296 (D) $2,721 
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Famil - ipsiiaia bdeeedineial cubehtaietans 
Mot 13 
Father ) 

Child, age 170 299 150 
16 and over | 


According to the table, how many AFDC family mesbers were re 


(A) 1,792 (R) 2,004 (Cc) 2,388 (D) 2,478 


Accordine to the table, the number of AFDC children, 16 and over, who were enrolle 


in IN in 1971, was greater than the number enrolled in 1969 by 


(A) 12 (B) 34 (C) 38 (D) 52 


Ace rdine to the table, the number of “FNC mothers who were enrolled in 197] 
increased over the nunver enrolled in 1269 most nearlv hy 


(A) 293%, (GB) 322% (C) 54 (D) 83% 


In 1972, if the number of AFDC mothers referred to ‘71% increases 5% over 1971 
number of fathers referred increases %, over 1971, and the number of children 
referred increases 5% over 1971, the total number of AFDC fanily members that will 
be referred in 1972 is most nearly 


1 
i) re 


(A) 1424 (RB) 1524 (“) 3139 (D) 3990 


Accordine to the table, the percentase of AFNC fathers not enro'’led in VIN in 1971 
\ 


of the number referred that year is most nearly 


(A) 23% (B) 25 co? ha (D) 77% 


63. 
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n ' Folic which wit ( t Jarseet por or tl budget f 

tire yur inistratic is 
(4) re 1 and rt rvic 
i'd, hem t 4 bi ul medic 
(C) t ! you 
Ey. vs t a) mid ior ’ < lopment 

cordi to tne latest statistics nublished by the U.S. Denartrent of Nealch, 
Pducatior ] Velfarve, the state whien spent tie largest amount of money per person 
for { lic ssista ; 
CX) Califorava (L) Massachusetts (C) Pennsylvania (D) New York. 


Accordins ta the most recent 1.8. Census Yureau Renort, the group livin: in 
’ 1 > 


iWew York City wnich has the lowest income level is the 


(A) Blacks (1.) Puerto Ricans (C) Doninicans (D) Haitians. 


he group that contains the largest number of individuals receiving public assistance 


ildren under wortineg age (C) the aped, disabled and blind 


Cc 
unemnloyed heads of families (D, unemployed single persons. 


major difficulty faced Ly new arrivals to cities since 1°59, which did not exist 
for earlier !uropean immigrants, is the f.ct that the majority of present-dav 
arrivals 


(A) it their native culture natterns 

(3) have an obviously darker skin color than mo.t lone-time residents 
(©) have little educatinra 

(0) have few occupational skills. 


must for 


Generally sneakine, low-income persons do not make maximum use of opportunities and 
services available to them, mainly hecause 


(A) most varaprofessional workers, while sincere in che desire to serve, 
are unable to reach the hard core 

(5) much of the routine paperwork in public assistance programs is now 
assicned to paraprofessional "0. kers 

(C) they have become increasingly self-reliant and prefer to cope with 
their problems without help 

(D) they lack the confidence and knc ior Necessary to make their needs 
known to the proper persons or avzenc'es, 


The one o} the following problems which has most recentiv become a sesious concern 
of the Youth Services Agency is the 


(A) increasing high school drop-out rate 
(S) resurrence of fighting youth gangs 
(C) spread of vouth narcotics addiction 
(D) lack of recreation programs. 


Of the following, the most recent development with regard to welfare recipients is 


(A) introduction of the declaration of need instead of an investication of 
eligibility 

(}) 4 major emphasis on employment prorrams 

(C) increased use ot casework therapy and psychiatric counselling 

(D) acceptance of nercotics addicts for ‘ome relief. 
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; 
69. Accoriing to a recent decision by a Fediial court, regular reporting at State 


70. 


Exployment Service offices to pick up checks or accest work can no longer be 
reyuired of vecipieuis of 


(A) Aid to the Disabled (C) Aid to Dependent Children 
(B) Home Relicf (D) Medicaid 


A basic objective of the proposal for revenue sharing currently under conetderation 


by *he U.S. Congress is to provide 


(A) state and local povernments with new scurces of revenue from the 
federal government and preater control over how this revenve is spent 

(B) the federal governneat with greater control over spending of certain 
federally-raised tax revenues 

(C) eafeguards against improper allocation of funds by state and local 
officiais and incentives to states for reporting violations by 
local government 

(D) a method of sharing federal revenue with the states and localities 
in accordance with their required expenditures for public assistance 
and social welfare services. 


GO ON TO NEXT PAGE 


ys 
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Feror a lye fume ) wth ie ss 9 ferer wr of the concept of parent 

j ] i a 

te. Ne f i 2] 1¢ t (fe) 

(3) . ay (i) T ice of anily Day Care 
ver called in t V'is besed in t follovine poverty area 

(A) South Ja i (C) Fast ev York 

(i) soutiy Pronx (D) Bedford-Stuvvesant. 

ihe apency called KARYOV-ACT is based in the followine poverty area: 

(A) Willi bure (C) Fast Marlen 

(3) Central Warler (ND) Lower Nast Side. 


Of the following, the organization which has waged an arpressive campaign to inform 
the poor of their legal rights is the 


(A) Legal Aid Society (C) American Civil Liberties Union 
(B) National Welfare Rights C sanization (D) National Urban Learue, 


Which one of the follovine programs is designed to meet the need for paraprofessionals 
in government agencies and institutions? 


(A) Concentrated Implovment Program (C) Opportunities Industrialization Center 
(B) Public Service Careers Proyram (D) Work Incentive Program. 


The popular name for the program authorized by Title I of the Demonstration Cities 
and ‘fetropolitan Development Act of 1966 is the Federal 


(A) Anti-Poverty Procram (C) Model Cities Program 
(B) Manpewer Development and Training Program (D) Revenue Sharing Program. 


An agency which currently sponsors sunportive programs to rehabilitate former 
harcotics addicts through full-time paid employment in such fields as park 
maintenance, rodent control, and messenger services is the 


(A) Volunteer Prison League (C) Judicial Council 
(B) Vera Institute of Justice (D) Daytop Village. 


Project “Manhood Foundation can best be described as an organization which provides 


(A) manpower services for former convicts aged 16 and over 

(B) supportive counselling for forner narcotics addicts 

(C) remedial education and vocational training for juvenile off ‘Ss 
(D) skills training for the disabled. 


The correct description of the functions of ASPIRA is that the organization provides 
disadvantaged youth of Puerto Rican background with 


(A) educational counselling, financial assistance and career planning services 
(6) vocational counselling and trainine in the skilled trades 

(C) remedial, rehabilitative aod recreational services 

(D) remedial education and vecational placement services. 


Which of the followine is a federal residential prorram that offers work experience, 
education, and skills training for disadvantaged youth from 16 to 21 years of age? 


(A) Trainine Resources for Youtn (C) Vocational Foundation Incorporated 
(B) Youth Umployment Service ()) Job Corps. 


83. 


£4, 


85. 


td . 
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ee foilowive, the bhasie adventare of »prarcra J ated te ces 
Ly orovidine federally subsidized regular ridd jo in public aac 
Luch # ororran vrovides 
(4) an opportunity fer the perticipants te Covelon and inmrove 
and prenare thenselves for non-subsia »1 jol wh the ecor 
situation improves 
(3) the participants with job security and frinee hencetits which 
nol available in private industry 
(C) public avencies vit’) hiphly motivated workers at Jow cost 
(PD) pvublie acencies vith funds to {111 vaca fes which would othe 


remain unfilled lecause of budeet restriction 


Assume that one of your job functions is to help develop employn 


for clients of the Rerional ‘Manpower Center. The one of the following 
he most useful in helping to identify likely industries for nlacene 


information on the 


(A) emplovers' attitudes 
(i) educational level of those emploved in the industries 
(C) nature of job vacancies 


Ssalarics of the clients before they became unemoldoyed. 


The agency whose overall operation is most Sinilar to that of thie 
Career Development Apency is the 


(A) Office of Labor Relations 

(RB) Division of Vocational Rehabilitation 
(C) State Employment Service 

(D) Youth Services Agency . 


\ standard reference source for manpower specialists developed bv t 
of Labor is known as the 


(A) Mental ‘feasurement Yearbook 

(5) Public Perso.nel Review 

(C) Classification and Compensation Schedule 
(D) Dictionary of Occurational Titles 


ar 


‘“anpower an 


he 


e 


u. 


nt would 


ich 


Department 


rT 
1 


yortunities 
vy} 


rnd 


The program which offers manpower services such as job development, placement and 
P 


referral to training for former convicts, including parolees, aped 
is called 


(KF) Project "lanhood Foundation : 
(C) Project Renewal 
(D) Vocational. Foundation Incorporated. 


(A) Opportunities Industrialization Center, Inc. 


16 
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t fol] the t it t re mW r re j ? 1 on job 
i {¢ ’ { i LG | it trie pment o ecor ic 
ic i t rch 4 tion Loe 

i und character of the unfilled de 1 ea r 

I it le 5 ( tine diseri lation inst minority gre nombers 

the € ! e uns { ) tic rou n ALET 

the o ies whic re i] t i t worke 


the follovwtas wer programs, tt 1¢ c eS not receive federal funds 
the 
Job Corps (C) “ork Incentive Progre 
Neighborhood Youth Corps (D) Emerpency Placevent Prorram. 
if lic Personnel Test is a widely used standardized test which airs to 


precict employment success. A study of an experimental srounp of disadvantaged 


vou 


€ 
showed that this group's 


lis 
Of 


th who were successfully emploved after particiration in a trainine rropran 
t 
t 


ores On the tesf were considerablv lower than those 


ted in the test manual as mininum vassinp scores for particular occunations. 
the following, the most approprixzte conclusion that can be drawn from tnese 


results is that the 


(A) 
(3) 


(C) 
(D) 


Job 
(A) 
(B) 


(C) 
(D) 


The 


test is not a valid test for the experimentel group 

experiment was poorly conducted ani should be repeated under improved 
conditions 

youths may actuaily be in need of additional rainine 

test is not useful in predictine on-the-job success for any eroun. 


restructuring can provide new careers for the disadvantaved by 


creating entrv-level positions and meanineful promotion opportunities 
for inexperier ced persons 

eliminating the more difficult duties cf a job 

lowering the educational qualificaticns for higher-level jobs 


training individuals in those aspects of a job 1n which thev are not 
skilied. 


New York Plan is d&@sicned to provide jobs and training for minority eroup 


members in the 


(A) 
(B) 
(C) 
(D) 


bufldiag and construction trades 
nt and needle trades 


ing and publishing trades 
-£ 


ront and warehousing trades. 


je" er 
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ANSWER SHEET COLLECTION: You are required to hapd in vour question boo} let, 

answer sheet, and all scratch paper after compleiiny the test. Aay candidate 
leaving the rcom with the question bootlet or serate: paper in his possession 
will be disqualified. When you have coupleted t! “st, remain seated and 
Summon the monitor. The monitor will pict up your answer sheet, question 

booklet, and seratch paper. Leave the building auickly and cuietly. You 
should make a note of your application number and include this auther in ; 
future correspondence with the Yepartnent of Personnel with respect to this 


examination. 


rt 
c 


Bs ys 


The Department of Personnei, in establishine the key answers to this test, re- 
Serves the right to determine which of the answers listed for each question is 
to be deemec and accredited as acceptable. The Department of Personnel further 


reserves the right to cancel and annul any question whenever, upon inquiry, it 
deems that none of the listed answe:s thereto can properly be consia red accept- 


able. 


KEY ANSWERS: The rating key for this test will be posted on the Of icial Notices 
Bulletin Board of hs Application Section of the Department of Personnel, 

49 Thomas Street, New Yor! » N.Y. 19013, for a period of three jays beeinnine 
October 25, 1972 


Please do not telephone the Department of Personnel requesting information as to 
the progress of the rating of this examination. Such information will not be 
available to candidates. All candidates in this examination wi” be notified 
individually by mail of the rating they receive when the rating of all candidates 
in the examination has been completed. Notify this Department promptly in writing 


of any change in your address to insure prompt delivery of such notice. 
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CovINGTon, HowarRpD, HaGoop & HOLLAND 
ATTORNEYS aT Law 
'S COLUMBUS CIRCLE,NEW YORK,N. Y. 10023 


G DONALD CCVINGTON 


EOWARD 0. HOWARD 22 265-9940 JAMES R. RHONE 
KENNETH D HAGOOD CABLE "COVINLAND, NEW YORK’ COUNSEL 
ALBERT HOLLAND 
ERM AES Lanne September 10, 1974 

Honorable Morris E. Lasker 

United States District Judge 

Umited States Courthouse 

Room 2903 

40 Foley Square 

New York, New York 10007 

Res dones v. H.R.A. — 73 Civ. 385 
Williams v. H.R.A. - 74 Civ. 91 


Dear Judge Lasker: 


This letter is in response to the question raised in the 
above captioned actions concerning the sufficiency of plaintiffs’ 
statistical case. 


As we understand it from conversations with Mr. Arditi, the 
question is whether the statistics before the Court with respect to 
some of the challenged examinations are sufficiently complete to 
permit the Court properly to draw inferences and make conclusions 
as to discriminatory impact. Specifically, the question put to us 
by Mr. Arditi is whether the Court may properly infer and conclude 
with respect to the challenged Open Competitive Supervisory HRS 
examination, where the ethnic identity and pass-fail rate of 51% of 
the examinees is known, that the ethnic identity and pass-fail rate 
of the other 49% is similar. We have been advised that the Court 
is considering obtaining "expert" opinion on the question of whether 
the "sample" is sufficiently representative. The issue has been 
raised in defendants' reply memorandum. The Court should be advised 
that, to date, we have not been able to find a statistician availa- 
ble to deal with the question. Parenthetically, we are not sure 
whether ‘t is a matter of securing one expert or more than one. More 
importaiily, we respectfully submit that the issue of the discrimi- 
natory iupact of the Open Competitive exam need not be reached. 


The challenged 0.C. examination is 87.5% identical to the chal- 
lenged promotional supervising HRS examination. The ethnic identity 
of the candidates on the promotional examination is fully known. 

The disparity of pass-fail ratios for Whites against Blacks and His~ 
panics is approximately 3-1/3 to 1. (The disparity for the ethnically 
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identified 0.C. candidates is a strikingly similar 3 to 1). As the 
Court is aware, appointments can be made from the 0.C. list only 
after the promotional examination list is exhausted - the 0.C. list 
in effect being a continuation of the promotional list. If the 
Court were to hold the "top" of the list, i.e., the promotional 
list, invalid, in that minorities were unconstitutionally excluded 
therefrom, it would, we submit, violate the dictates of both common 
sense and the law to permit appointments to be made from the 
"bottom" of the list, i.e., the 0.C. list. Such appointments would 
just as effectively defeat the right of the minorities who failed 
the promotional list as would appointments from the promotional 
list itself. 


If the Court wishes argument on the question or more formal 
Papers, we will be pleased to respond. We trust, however, that the 
Court will accord due consideration to what is set forth herein 
without the necessity of additional formal proceedings. 


We thank the Court for its attention to this letter. 
Respectfully, F 


Fd 


af 
2 (one 
¢# EDWARD O.“HOWARD 


EOH: aw 

cc: Ms. P. Omansky 
Ms. D. Greenberg 
Mr. J. Jones 


Ms. K. Ivey 
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UNITED STATES DISTRICT COURT 
SOUTHERN DISTRICT O” NEW YORK 


UNITED STATES COURT HOUSE 
NEw YorK, N. Y. 10007 


CHAMBERS OF 
JUDGE Morris E. LASKER 


September 12, 1974 


Edwe O. Howard, Esq. 

Cov: «;ton, Howard, Hagood & Holland, Esqs. 
15 Columbus Circle 

New York, New York 10023 


Paula J. Omansky, Esq. 
Assistant Corporation Counsel 
Municipal Building 

New York, New York 10007 


Jones vv. HORA. 73: ¢iv.. 3835 
Williams v. H.R-A. 74 Civ. 91 


Dear Mr. Howard and Ms. Omansky: 


Judge Lasker has Mr. Howard's letter of Septem- 
bex 10th. It properly states the considerations which 
the court has in mind. 


Judge Lasker has instructed me by this letter 
to request Ms. Omansky to write him commenting on the 
observations made in Mr. Howard's letter of September 


10th. 
Very truly wie 
Ralph Arditi 
Law Clerk tec Judge Lasker 
RA/cw 


cc: Deborah Greenberg, Esq. 
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September 24, 1974 


Hon. Morris E. Lasker 

United States District Judge 
Southerr District of New York 
United States Courthouse 
Foley Square (Rm. 2903) 

New York, NY 10007 


Re: Jones v. H.R.A. - 73 Civ. 
3815 


Will 8 Vv. H.R.A. - 74 
iw. 91 


Dear Judge Lasker: 


I am writing in response to Mr. Arditi's letter 
asking me to comment on Mr. Howard's letter of September 
10, 1974, which Z received on September 16. Since I do 
not think that Mr. Howard's letter was responsive *o the 
issue as posed by the Court, I take this opportunity to 
set forth my understanding of the problem. 


On my return from vacation, Mr. Arditi told me 
that the Court thought that defe iants' Memorandum of Law 
had raiseé “thorny problems" about the adeguacy of 
plaintiffs’ proof on their statistical case. Therefore, 
the Court on its own motion wanted to re-open the record 
for purposes of secur — opinion on the significance 
of the raw data, specifical Y, whether it can be ferred 
from the incomplete data on 2 of the 3 challenged open 

tive exams, for Sup. HRS and Senior ERS xespectively, 
that blacks, whites, and Hispanics would be likaly to pases 
at the same rates as they did on the two promotional exams 
given for those positions. (Parenthetically, I would 
aote that the problem of the open competitive exam for the 
lower level position of HRS remains. There, plaintiffs 
did not challenge the promotional HRS exam, the raw data 
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tending to disprove any disparate impact.) 


The Court suggested various options to the 
parties: (a) an expert that both sides would agree 
upon; (b) an expert from each side; or (c) an expert 
appointed by the court. 


Notwithstanding defendants' original misgivings 
as to the propriety of re-opening the record 7 months 
after trial when plaintiffs had not so moved, we came to 
agree, for several reasons, that the court was correct 
to ca)l for expert opinion cn this crucial aspect of the 
case and we had already made arrangements to secure such 
opinion. 


In the meantime, Mr. Howard wrote the letter 
of September 10, 1974, declining all of the options 
presented by the Court. 


To my kno-,ledge, the Court has not withdrawn 
its request to re-open the record for expert opinion 
and I am *omewhat confounded by Mr. Arditi's statement 
that Mr. Howard's letter “properly states the considerations 
which the court has in mind." 


Defendants propose to submit expert opinion by 
way of affidavits on the problem posed by the Court. 
Accordingly, we ask that the Court set a firm date during 
the week beginning September 30 for final submission of 
papers by both sides. 


In addition to the unavailability of a 
statistician, plaintiffs justify their refusal to submit 
any expert opinion on the ground that "the issue of 
the discriminatory impact of the Open Competitive exam 
need not be reached.” This ergument, offered in response 
to the court's call for expert testimony, was never before 
raised, and represents a siqnificant departure from 
plaintiffs' original positiun in this case. Defendants 
submit that the argument has no merit in fact or in law, 
and in fairness, we should have the opportunity to reply. 
However, in order to put an end to the dialogue between 
the lawyers, we propose to submit our response to the new 
legal argument on th~ same date the Court fixes for final 
submission of expert © pinion. 


Very truly yours, 


Paula J. Omansky 


cc: Edward Howard, Esq. 
Deborah Greenberg, Esq. 


AT \ 


> 
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Letter of Paula J. Omausky Dated October 10, 1974 


LAW DEPARTMENT 


MUNICIPAL BUILDING, NEW YORK, N. Y. 10007 


io 


ot. 
“New ork , PAULA J OMANSKY 


Executive Assistant Corpoie on Counsei 


October 10, 1974 


Hon. Morris E. Lasker 

United States District Judge 
Southern District of New York 
United States Courthouse 
Foley Square - Room 2903 

New York, NY 10007 


Dear Judge Lasker: 


Ip their letter of September 10, 1974, 
plaintiffs disclaim their responsibility to make a 
prima facie case on the open competitive exams. In a 
sudden shift in strategy, they “submit that the issue 
of the discriminatory impact of the open competitive 
exam need not be reached." Plaintiffs reason that the 
promotional list, for which the statistics are fullv 
known, represents the "top" of a list established for 
the position of Supervising Human Resources Specialist; 
and if a discriminatory impact is found there, it would 
"violate the dictates of both common sense and the law 
to permit appointment to be made from the ‘'bottom' of 
the list, i.e. the 0.c. list." 


What plaintiffs have done is to seize on a 
provision of State law according different treatment to 
two lists and then obliterate that very distinction by 
meraqing the two lists into one. Their appeal to reason 
falls short of both the facts and the law. % 


To begin with, the facts are tHat there are 
two separate lists established as a result of two different 
exams administered to two different populations. If the 
promotional list were the "top" of one list, with low 
scoring promotional candidates ranked higher than high- 
scoring open competitives, then the State Constitution's 
mandate that appointments be based on merit and fitness 
* suid be violated. 
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In designating the promotional list as the 
"top" of the list, plaintiffs suggest that these 
candidates deserve prior appointment. However, the State 
law requirement that the promotional list be exhausted 
before any appointments are made from the open competitive 
nas nothing to do with superior merit. Rather, it 
represents a legislative determination to reward a quality 
other than straight exam performance, namely prior civil 
service employment. Common sense does not dictate the 
prior appointment of the promotional candidates, as 
plaintiffs observed curing the litigation. 


We note that common sense did not impel 
plointiffs to challenge the promotional HRS exam. Yet we 
have precisely the same result which plaintiffs urge this 
court to avoid, only in reverse. The "top" px. motional 
candidates are assured of appointment although the *bottom* 
of the list may have shown that an exam, "87.5% identical" 
was unconstitutional. Neithe:> the legislative preference 
shown to promotional candidate 10r common sense dictates 
this result. 


The promotional list would be appointed first 
even if the exams were totally dissimilar. In this case, 
the two exams were sim.°ur, and plaintiffs rely heavily 
on the similarity. .veu if the two examinations were 
identical, but administered to two different populations, 
the results for the known population would be of limited, 
if any, value in predicting the results for the unknown 
population unless there was statistical assurance that 
it was proper to do so. This is the problem which 
plaintiffs seek to avoid. It begs the question to note 
the similarity of the exams and to ignore the statistics. 
"Numbers comprise a critical element of what the case 
purports to be about." Gonzalez v. City of New York, 

4 E.P.D. §7867 (S.D.N.¥. I97Z). 


We may, for the sake of argument, assume that 
plaintiffs' hypothesis is correct, that there is one 
list and the promotional candidates are the "top" of that 
list. Defendants submit that plaintiffs are worse off 
than when they started, since they are then left-with 
incomplete statistics even for one examination. The 
case law does not permit such an incompléte statistical 
showing. 


Plaintiffs offer no guidance as to how the 
isolated HRS open competitive is to be treated if their 
theory is accepted. Logically, plain+‘ffs should tzke 
the position that it is equally unnecessary to reach the 
issue of statistics on the open competitive HRS where 
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the promotional results are known, even if unfavorable 
to them and thus unchallenged. Plaintiffs’ theory 
cannot be conditioned on whether or not they have chosen 
to challenge the promotional exam. 


If it is necessary to reach the issue of the 
disparate impact for the Open competitive HRS exam, the 
same should be true ror the two other open competitives. 
Otherwise, a difference in treatment would turn on 
whether the promotional figures favored or disfavored 
plaintiffs, an untenable result. 


Plaintiffs have offered a new theory as an 
alternative to submitting expert Opinion on the 
Significance of the Statistics already in evidence. UWe 


urge the Court to reject it as contrary to both the 
facts and the law. 


Very truly yours, 
>) 8 . 
\ ual } Muenely 
/ 
Paula J. Omansky 


cc: Deborah Greenberg, Esq. 
Edward Howard, Esq. 
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m v 
UNITED STATES DISTRICT COURT 
SOUTHERN DISTRICY OF NEw York 
UNITED STATES COURT HOUSE 
New YorRK, N. Y. 10007 


CHAMBERS OF 
JupGce Morrie E. Lasker 


October 21, 1974 


TO: Edward 0. Howard, Esq. 
Deborah Greenberg, "sq. 
Paula Omansky, Esq. 


RE: Jones v. H.R.A. 73 Civ. 3815 
Williams v. H.R.A. 74 Civ. 91 


Judge Lasker has Ms. Omansky's letter of October 
10th, which was written at his request in answer to Mr. 
Howard's letter of September 10th. 


After considering the arguments presented in 
each letter, Judge Lasker requests that the plaintiffs 
include in the expert opinion evidence which they have 
been directed to submit, an opinion as to whether in 
the case of examinations in regard to which the ethnic 
makeup Of a group of examinees is partly known, there 
is evidence of record from which conclusions as to 
the ethnic makeup and pass-fail rate of the unknown 
portion of the group can reasonably be inferred. 


Very -ruly yours, _ 
( 


tp—h-—\. 


Ralph Arditi. 
Law Clerk to Judge Lasker 


RA/cw 
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Affidavit of Richard S. Barrett 
UNITED STATES DISTRICT COURT 


SourueErN Districr or New York 


73 Civ. 3815 


—————— 
JAMES C., Jones, et al., 
Plaintiffs, 


—against— 


Tue New York Crry Human Resources 
ADMINISTRATION, etc., et al., 
Defendants. 
74 Civ. 91 
eltiertsatinctivaatatalllie ta ai ie tle 
Dorotuy Wituiams, et al. 
Plaintiffs, 
—against— 


THe New York City Human Resources 
ADMINISTRATION, @éfc., et al., 


Defendants. 
OO Se 


STATE OF NEW JERSEY, 
County or Hupson, ss.: 


Ricnarp S. Barrett, being duly sworn, deposes and says: 


1. I make this affidavit at the request of plaintiff’s coun- 
sel in the above-styled case. IT am presently Direetor, Lab- 
oratory of Psychological Studies and Professor and Director 
of the Applied Psychology Division of Stevens Institute of 
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Technology. My current vita is attached hereto as Exhi- 
bit A. 

2. In addition to the background deseribed in Exhibit A, 
[ have taken courses in elementary, intermediate and ad- 
vance statistics, and I have taught elementary statisties as 
applied to psychology. In my practice of psychology, I reg- 


ularly use statistics. 


3. The Chi-Square Test is a simple, commonly used and 
generally accepted means of analyzing the kinds of sta- 
tistics supplied in this case. It ean be used with each sub- 
, (1) whether he 
passed or failed the test, and (2) whether he falls into the 
racial/ethnic category of white or minority. Two minority 


ject as classified in two categories, i.e. 


classes were used in this study, black, and black and His- 
panic combined. (There are too few Hispanies for analysis 
of whites versus Hispanics). The few subjects who were 


classified as “other” are not included in this study. 


4. To understand the Chi-Square Test, one may consider 
the situation in which there are 100 blacks and 100 whites, 
and of this group, there are 80 who pass the test, and 120 
whe * 1. If the test were absolutely even-handed, we could 
expect that 40 blacks would pass, and 40 whites \.vuld pass. 
I, real life, we are almost certain to observe that not ex- 
actly 40 from each group pass the test. The purpose of the 
Chi-Square Test is to determine whether the observed pass- 
fail rate for the two groups differs so much from the ex- 
pected rate that we are reasonably sure that the differences 
stem from real differences in the two groups rather than 


from random differences arising from chanee variation in 
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the sample. It is almost inevitable that there will be some 
observed difference between the two groups; the question 


is whether the observed difference is so great that no 
reasonable person would assert that the two groups are 


really the same, and that the differences arise only from 
sampling error. 


d. In preparing this affidavit, I have computed the fol- 
lowing Chi-Squares,* with the following results: 


Examination Comparison Chi-Square 
1631 Black v. White 17.81 
Minority v. White 19.63 
2013 Black v. White 76.40 
Minority v. White 80.42 
1626 Black v. White 44.60 
Minority v. White 43.65 
1099 Black v. White 59.40 
Minority v. White 62.49 
1097 Black v. White 11.49 
Minority v. White 15.32 
1625 Minority v. White (=) 


J. A Chi-Square of 6.64 will oceur less than one time in 
100 as the result of chance fluctuations in the sample. Con- 


* The Chi-Squares for Examinations +2013, 1099 and 1097 were 
computed only with respect to the candidates whose racial or ethnic 
identity is known. 


** The nature of Chi-Square dictates that when the expected 
number of cases in one cell falls below 5, the test should not be used 
because the results are too unstable to be relied upon. In this in- 
stance, the expected number of cases to fall in the cell Whites/Pass 
is 1.21, and in the cell Whites/Fail is 3.79. Therefore, Chi-Square 
was not computed. 
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ventional statistical tables do not include velues as large 
as those reported above because their occurrence as chance 
events is too small to be considered seriously. The con- 
clusion to be drawn is that there is a genuine difference in 
the pass rates of the two groups. 


7. In some tests, there is a substantial number of can- 
didates whose racial or ethnic identity is unknown. Of 
those who took Examination 2013, 486 of 994 are not iden- 
tifiable by racial or e.unie group; of 683 who took Exam- 
ination 1099, 277 are not } tifiable; and of 606 who took 
Examination 1097, 278 a». not identified. The question that 
is not resolved by the statistical analysis reported above is 
whether the data on the entire group would show the same 
result as is shown on the vrovy for whom the race or eth- 
nicitv is known. Strict'y speaking, such a determination 
can be made only if the: is reason to believe that those 
whose identity is not k:own are a random sample of the 
total group. There is of course, no way to make this deter- 
mination. However, the size of the Chi-Square statistics 
reported above is so yreat thr< hose whose race or ethnicity 
.s unknown would have to ciffer in an unrealistically large 
degree froi those whose ‘leatity is known to lead to the 
conelusion that the tesis aie free of adverse impact. In the 
abs nee of compelling evidence that those of unknown iden- 
tity ave from a totally lifferent group than those whose 
identity is known the results of the above analysis must 
apply to them as well. 


8. I have examined Examination #2013 and 1097. T am 
informed that Examination #1099 is made up of items 


from these two tests. In my opinion, these tests are made 
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up of items of the type on which blacks and Hispanics gen- 
erally do more poorly than whites. 


3. My summary opinion based on an examination of the 
data and the tests themselves is that Examination #163), 
1097, 2013, 1626 and 1099 hav a substantial adverse impact 
on blacks, and on minorities, i.e., blacks and slispanics 
combined. 


Ricuarp S. Barrerr 


(Sworn to November 4, 1974.) 
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BARRETT, RICHARD S, Born, 1924, Marricd, 3 Children 


5 Rverview Place 
Hastings-~on-lNudson, N.Y. 10706 


Educational Rackrround: 


1948 
195) 
1956 


1973 
to 
Date 


1972 
io 
1973 
197) 
to 
Daie 


1971 


to 
(1972 


1971 


1969 


Work 


= BOS. ministrative Engineering, Cornell University 
- M.A., fucation, Syracuse University 
- Ph.D,, Industrial 'sychology, Western Reserve University 


History 
Dircetor, Laberacory of Paycholorienl Studies: Professar 
and Director of Applied Psycholopy Division, Stevens 
Instiiute of Technolory i 


Visiting: Pre ossor of Menagement Science, Stevens 
lustitute of Vechnology 


. 
Consultant on fair employment, Expert witness in Grips 
Vv. Duke Power, Chanee ve. Roard of Taauniners, Bridgeport 


Guavurane: ve Red dpeport, Casiro wv. Reecher, U,S.v. 


Director, Docet Corporation, Develop educational progran, 


Director, Project CONCEPT, City University of New York, 
Coordinate cooperative education programs in New York City 
Assistant Director for KNducational Research, Center for Urban 
Education, New York. Direct rescarch on education of urban 
children, : 


Senior Associate, Case and Company, Ine. Consult in personnel 


and industrial psychology, cducation, fair employment practices 


Director, Materials Evaluation, Science Research Associates 
Supervise investigations in education, 


Research Associate Professor of Psychology, New York University. 
Principal Investigator "Differential Selection Among Applicants 


Yrom Different Socio-Econowic and Ethnic Backgrounds..." 


Assistant to Associate Professor of Management Engineering and 
Psychology, Program Director, Research Center for Industrial 
Bohavior, New York University. Supervise research in applied 
psychology and consult. 


a 
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1955 Vice-President of Personnel Research and Development Corp., 
to Cleveland, Ohio. Consulted in industrial psychology. 
1958 © 
Earlier Research Associate, Operations Research Group at Case 
Posi-~ Institule of Technology. Professional Associate, Richardson, 
tions Bellows, Wenry & Co., New York, Industrial Engineer, Liberty 


Products Corn,, Farmingdale; New York. 
Professional Sucieties: 

American Psychological Association 

Eastern Psychological Association 

Amevican Educational Research Association 

New York State Psychological Association 

_ Metropolitan New York Association for Applied Psychology 

Honorary Sociotios: 

Phi Delta Kappa 

Alpha Pi Mu 
Certification; 

Certificd Psychologist, New York Stato 


‘Diploma: 


Diplomate in Industyial Psychology, American Board of 
Professional Psychology 


Community Activities: 
Trustee, Board of Education, Dobbs Ferry, 1962 - 1965 
Selected Publications: 
with Taylor, E.K,, Parker, J.W,, and Martins, L. Rating... 
Scale Content I. Scale Information and Supervisory Roetings. 


Personnel Psychology, 1958, 11, pp. 333-346, 


with Taylor, E.K,, Parker, J.W., and Martins, L. Rating 
Scale Content II. Personnel Psychology, 1958, 11, pp. 519-534, 


with Parker, J.W,, Taylor, E.K,, and Martins, L. Rating: 
Scale Content IYI. Relationship Between Supervisory-and 
Sel{-Ratings. Personnel Psychology, 1959, 12, pp. 49-64, 
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Selected Publications:  (Cont'd.) 
Survey of Human Engincering, Machine Design, January 24, 1957, 


The Agreement Seale: A Preliminary Report, Personnel 
Psychology, 1961, 14, pp. 151-165. 


with Goldbers, Mo. and Dawson, R.I., Comparison of Programned 
and Conventional instruction Methods, Journal of Applied 
Psychology, L665, 46, pp, 110-114. 


with Feinberg, M.R., Raliinsky, B., Garber, R., Wich.) ok... 

aid Steckhel, L. Outside Consultants to Industry: Strengths, 
Problens, and Pitfalls (A Symposium). Comments on the 
Symposium, Personnel Psycholopy, 1964, 2, pp. 126-133, 


With Heinneman, R.FLD. Human Engineering Considerations in 
the “valuation of the Instructor's Console of OFT 's. Report 
wo Special Devices Center, 1953, 


With Katzell, R.A,, and rarncr, TC; doh Satisfaction, Job 
Pertormance and Situationh] Characteristics, Journal of 
Applicd Psychology, 1961, 4b, pp. 65-72, ; 


The Process of Predicting Job Performance, Personnel Psychol- 
ogy, 1958, 11, pp. 39-57, 


Guide to Using Psycholoical Tests. MJarvard Business Review, 
1963, Sept. - Oct., pp. 138-146, 


Performance Suitability and Role Agreement: Two Factors 
Related to Job Satisfaction, Personiucl Psychology, 1963 
Winter, pp. 345-357, 


Explorations in Job Satisfaction and Performance Rating, 
Personnel Administration, Sept, - Oct., 1964, 27, pp. 14-17. 


The Impact of the Executive Program on the Participants, 
Journal of Industrial Psychology, 3, 1965, pp. 1-14, 


Performance Rating. Chicago, Science Research: Associates, 1966, 
Problems de selection, Synopsis, March 1965, pp. 3-16, 


Reviews in Buros, 0.K, Sixth Mental Measurements Yearbook, 
Highland Park, Gryphon Press. ; i 


The Influence of Supervisor's Requirements on Ratings. 
Personnel Psychology, 1966 Winter, pp. 375-387. 


& 
as : ? 
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Selected Publications: (Cont 'd,) 


Gray Areas in Black and White Testing, Harvard Business 
Review, January-February, 1968, pp. 92.-95, 


wi.’ Kirkpatrick, J.J,, Ewen, RB, and Katzell, RA, 
Testing and Fair Employment, New York, New York University 


Press, S065; 


The Conpnicr Mentality, Phi Delta happan. April, 1968, 
pp. 431-4355, 


with Ifeller, Rarbara Nn. Expand and Improve: A Critien] 
Review of the First Three Yours of HS) )\ Matle’ Uo in New York 


City, hew York, Genter for 


Urban Lducati on, i970 


the Expert Witness: Diserimination in Muployment, Professional 
Psycholozy 2, 1971, pp. 366-372. 


"View from the Under Side, The Industrial Msychologist 
(In Press) . 
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UNITED STATES DISTRICT CQURT 
SouTHERN Districr New Yo: 
73 Civ 3815 


OS 
JAMES ( oS, GLoriA DeBerry, et al., individually and 
ou. vehalf of all others similarly situated, 


Plaintiffs, 


—against— 


THe New York Ciry Human Resources ADMINISTRATION, 
JuLE M. Sucarman, et al., 
Defendants. 
OE 
STATE OF New York, 
County or New York, ss.: 


Gus W. Gramaas, being duly sworn, deposes and says: 


1. lam presently an Assistant Professor of Management 
Science at Columbia University, Graduate School of Busi- 
ness in New York, in the area of Staticties and Operations 
Research. I teach Multivariate Statistics, Advanced Sta- 
tistics, and Elementary Statisties. 


2. I was graduated in 1963 with an A.B. degree in Mathe- 
matics from Columbia College and in 1972 received a Ph.D. 
degree in Mathematical Statisties from Columbia Univer- 


sity. 
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3. I have been teaching at the Graduate Sc..ool of Busi- 
ness since 1969. In 1970 and in 1973, I was one of 3 recipi- 
ents of the Graduate School’s Distinguished Teaching 
Award. 


4. My professional memberships include the Operations 
Research Society of America; the American Statistical 
Association, and the Institute of Mathematical Statisties. 


0. Iwas consulted by Paula Omansky of the Corporation 
Counsel’s Office and asked to give my professional opinion 
on the statistical significance of certain data 

The figures which were supplied to me and on which my 
analysis is based are attached hereto as Exhibit A. I was 
advised that examination Nos. 1631, 1626, and 1625 are 
known as promotional exams and were open only to City 
employees working for the Human Resources Administra- 
tion (HRA). Examination Nos. 2013, 1099, and 1097 are 
known as open competitive examinations and were open to 
all persons who met the City’s eligibility requirements. I 
am further advised that the promotional and ypen competi- 
tive exams for Supervising Human Resources Specialist 
(HRS), and Senior HRS, and Human Resources Specialist, 
differed by only 10 questions at each level. Racial identifi- 
cation was determined from the files of HRA, and noted 
on the computer runs of the results of the examination. 

The racially identified population will be known as the 
“HRA” population. The population for which no racial 
identification was made will be known as the “non-HRA” 
population. 
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6. Specifically | was asked whether the racial make-up 
and pass-fail rates for unidentified test-takers could be 
inferred from the figures available for the known test- 
takers or whether an examination for which » ‘1 results 
were known could be said to have had a racialiy disparate 
my » unidentified persons when no results were known 
as te 

It is my opinion that neither racial break-down nor 
past-fail can be inferred about the unidentified test-takers 
from th: known group and that there is no statistical basis 
for postulating a .acially disparate impact on the uniden- 
tified group based on the results available for the known 


group. 


8. On the other hand, a set of assumptions could be made 
about the non-HRA population which could then be tested 
against the only known fact about that population, namely 
the over-all pass-rate. The issue ‘hen becomes whether the 
known fact is consistent or inconsistent with our assump- 
tions. 


9. For example, consider the figures available for exam 
No. 2013. We know nothing about the non-HRA ponulation 
except the over-all pass-rate, 183/486 or .377. We save no 
figures at all on the racial breakdown and pass-fail rates 
even of a sample cf that population. 

For the sake of argument, I will make three assumptions 
which plaintiffs would need to make in order to establish 


disparate impact on the non-HRA population. 
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1) that the non-HRA population breaks down into the 


same racial groups as the HRA population; in 
other words 


Blacks comprise .494 (247/500) ; 
Whites comprise .466 233/500) ; 
Hispanics comprise ()' (20/500) 
2) that the pass-fai) >tes +e the se ne for both HRA 


aad non-HRA popux Jas; in other words 


Blacks passed at the rate of 16% (39/247) 
Whites passed at the rat of 54% (125/233) 
Hispanics passed at the rate of 15% (3/20) ; 


9 
I~ 


3) that the 486 non-HRA candidates are a random 
sample of the total population which was eligible 
to take the open competitive examination. 


I emphasize that these assumptions are matters of pure 
speculation on my part. I have no personal knowledge that 
the assumptions are even reasonable and I am advised that 
“er? 1: > no facts in the record of the trial to warrant these 
acsumptions. 


10. Nonetheless, based on these assumptions, 


Blacks compr::e °10 of the non-HRA group 
(486x.499) 


Whites comprise 227 of the non-HRA group 
(486x.466) 


Hispanics comprise 19 of the non-HRA group 
(486x.04) 
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We could then expect an over-all pass rate of .337 (164/ 


486) : 
Total Pass 
PRBORG «Si desnccscss 240 38 (240x.16) 
WHOS cece, 227 123 (227x.54) 
Hispanic ............ 19 3 (19x.15) 
486 164 


The real pass rate is .3877 (183/486) 


11. The statisticai question is whether the observed or 
real pass rate (.377) is significantly different from the 
hypothesized rate (.337). I conclude that it is not, based 
o1 standard statistical formulas: 


0 prop=  /.377(.663) = /.2234 = /.00040=.0214 
: 486 486 
Z= .337-.377 = .04 = ~1.869 
-0214 0214 


The two formulas used are standard statistical proce- 
dures when we have a random sample from a population 
and we are testing an observed proportion versus a hy- 
pothesized proportion. 

The sigma formula (0— prop) is a measure of variation 
(a type of standard deviation) in the hypothesized popu- 
lation. The Z formula compares the two rates with respect 
to the measure of variation and allows us to test statis- 


~« 
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tically whether the observed rate is significantly different 
from the hypothesized rate. 

In this hypothetical example, as I have stated the rates 
are not significantly different. More specifically, at the 
95% level of eonfidence (the standard level at which we 
reject or ace ‘ hypothesis), meaning that we are willing 
to take onl, 5% chance of being wrong, we »duld acc’ t 
that the hypothesized and observed pass-rates are the sa. .e. 


12. I emphasize that the statistical data analysis we have 
just performed does not establish that the three assump- 
tions we made are correct. The analysis merely means that 
if we are willing to make the three assumptions for exam 
No. 2015, the observed data, namely the real pass-rate of 
.377, does not contradict the three assumptions. 


13. However, this is true only for exam No. 2013. If 
we made the same assumptions for exams No. 1099 and 
1097, as we made for exam No. 2013, the observed data, the 
over-all pass-rates, do in fact contradict the assumptions. 

For exam No. 1099: 

1. Assume same population breakdown: 
Black == 221/406 = .544 
White = 155/406 = .382 
Hispanic = 30/406 — .074 
2. Assume same pass-fail rate: 
Black = 56/221 = .253 
White = 101/155 = .652 
Hispanic = 8/30 = .267 
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3. Then from a random sample of 277 of the non-HRA 
group, we would expect an overall pass rate of .404 
(112/277) 


Total Pass 


LLY) cee eee te 151 3 (151x.253) 
MUNIN ciatceccsnneds 106 69 (106x.652) 
Hispanic ............ 20 3 (20x.267) 
The real or observed pass rate is 90/277 = .325. Thus: 
0 prop = /.404(596) = /.000869 = .0295 
Pay By 
Z = ~404-.325 = 2.678 
.0295 


14. The rates are significantly different at the 95% level 
of confidence and we would reject the hypothesis that the 
rates are the same. The observed pass-rate contradicts at 
least one of the assumptions we were willing to make. I 
am unable to state which assumption is invalid and there 
is no way to establish which one is invalid from the date 
in Exhibit A. 


15. Similarly, for exam No. 1097 


1. Assume same population breakdown: 


Black = 175/326 = .537 
8 White = 115/326 = .353 
F Hispanic = 36/326 = .110 
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bo 


Assume same pass-fail rate: 

Black = 55/175 = .314 

White = 59/115 = .513 

Hispanic = 7/36 == .194 

3. Then from a random sample of 278 of the non-HRA 
group we would expect over-all pass rate of .371 


(103/278) : 
Total Pass 
MN ie a ncakan secs 149 47 (149x.314) 
WHT G0 Soe 98 50 (98.513) 
Hispanic ............ 31 6 (31x.194) 
The real rate is 78/278 = .281 
Thus 
0 prop = /.371 (.629) = /.00089 =.0289 
278 
and 
Z = As oe 1 = 3.114 
-028 


The rates are significantly different at the 95% level of con- 
fidence. The data again contradicts at least one of the 
assumptions. 


16. I could make any number of assumptions about the 
non-HRA group since the only known fact is the over-all 
pass rate. For example, I could assume that each group 
in the non-HRA population for exam No. 2013, minority 
and non-minority alike, had an over-all pass rate of .377, 
identical to the observed pass rate. A similar statistical 
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analysis would not contradict this assumption either. On 
the other hand, it does not establish that this assumption 
is correct. 

17. ‘There is no statistical basis for preferring one set 


of assumptions over another in the absence of facts. 


IS. The only valid statistical conclusion that can be 
drawn from the data on exam No, 2015 is that .377, the 
observed pass rate, is a good estimate of the over-all pass 


rate for the non-HRA group. 


19. In order to make valid statistical analysis of the ef- 
fect of exam No. 2013 on the non-HRA group, we would 
need to know the racial identity of the examinees. This 
should be verified independently, presumably at the time 
at which they appeared for the exam. Then, the pass-fail 
rates for the different racial groups could be compared. 

It should be noted that this is substantially the data 
which is available for the HRA employees. Based on this 
data, I might be tempted to say that disparate impact re- 
sulting from the examinations has been established for the 
HRA population. Yet on closer examination, disturbing 
inconsistencies are revealed. For example, while 2 out of 
the 3 examinations for Senior HRA (Nos. 1626 and 1099) 
had disparate impact, a third (No. 1094) apparently did 
not. This is also true for the Human Resources Specialist 
Ixaminations (compare No. 1625 with Nos. 1095 and 1097). 

Pusccem ore, the Hispanic HRA population does not 
have a consistent pattern with relation to the Black HRA 
population. At times, their pass rate is much higher (Nos. 


1095 and 1626); at times, much lower (No. 1097). 
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Because of these inconsistencies, the data is not conclu- 
sive that there is disparate unpact on the HRA population. 


20. I have been asked (1) whether any meaningful sta- 
tistical conclusions can be drawn about a group if less than 
the complete data is available with regard to racial com- 
position and pass fail rates; (2) why, on exam No, 2013, 
for example, the HRA group, which represents 51% (508/ 
994) of the total exam taking population, is not a sufficient 
or reliable sample of the non-HRA population from which 
meaningful statistical conclusions may be drawn, and (3) 
what significance, if any, can be attached to the fact that 
the HRA group passed the open competitive exam (No. 
2013) a* rates almost identical to the HRA group which 
took the promotional exam. 


21. As to the first question, statistical onclusions can 
be drawn if we have less than complete data on a popula- 
tion. However, we do not have here sufficient facts to draw 
any conclusion. For example, in a market test sample, you 
would want to find out what percentage of the population 
is interested in buying a product. As a statistician, you 
would draw a random sample from the population in which 
you were interested and send out questionnaires. A typical 
response rate if 20%. To draw any statistical references 
from the 20% response group, you would have to do follow- 
up studies among the 80% non-response to settle any ques- 
tion of bias in the statistical sense: is it a characteristic 
of the non-response group that they were not interested in 
the product? If as a result of the follow-up study, you find 
that there is no bias, you can proceed to base estimates on 
the data which you do have from the 20% response group. 
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Here, the HRA group is not a random sample of the 
non-IIRA group. In fact, we know nothing of the non-HRA 
group in terms of racial make-up and pass fail rates. Most 
assuredly, we have no follow-up studies on the non-HRA, 
or non-response group. 

The data which we do have on the HRA group is of no 
greater value in drawing inferences about the non-HRA 
group than it would be for drawing inferences about an 
examination taking group who were all residents of 


C ’:osh. 


) 


22. As for question 2), as I have stated above, the HRA 
group is not a sample of the non-HRA group. Therefore, 
we need not reach the question of whether it is a sufficient 
or reliable sample. 

23. As for question 3), there is no significance in the 
fact that the IRA group passed the open competitive exam 
for Supervising HRA (No. 2013) at rates almost identical 
to the HRA group which took the promotional exam (No. 
1631). One would expect this result because it is the same 
group, HRA employees, and almost the same exam. It 
provides us with no information about the non-HRA group. 


In summary, then, the racial make-up and pass-fail rates 
for the unknown non-HRA group cannot be inferred from 
the data available for the known HRA group and there is 
no statistical basis for postulating a racially disparate 
impact on the unidentified group based on the results for 
the HRA group. 

Gus W. GramMas 


(Sworn to November 7, 1974.) 
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Exam Wo. 163 ( tiona 
Pas Pail Total Sin 
Bla RZ Si 69 17 
Whit 28 24 52 542 
Hispanics “ 13 LG ao! 
Other - tk a 0% 
43 95 138 
u 26) (Op. ¢ t 
Pass Fail Total &si 


Blacks 39 208 247 16% ‘ 
| 
| 

Whites 125 108 233 543 

Hispanics 3 17 2u 15% 

Other ee a ents 63% \ 

172 336 508 
| 

Unknowns 183 303 48¢ 382 : 

355 639 994 


Lav 
What 
1): ni 
Uther 
Blac 


Unknowns 


Blacks 
Whites 
Hispanics 


Other 


Unknowns 
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‘ ) | 15] 

i lt (1 Lie ) 

i raid rotal 6 Pa 
] 5] 62 18 
30 4 34 BE 

3 5 8 38 


44 Ge 196 

1099 ( n Com tive) 
Pa Fail Total ~=—& Passi 
16 165 22] 25 
10) 54 155 65% 

8 22 _ 30 27% 
165 24) 406 
; 90 187 2 17 32% 
255 428 683 


1094 (iDT-Open Conpetitive) 
wider he Bigmdacct tbh AM 


18 al 59 31% 


18 78 96 19% 
48 145 193 


% Passing 
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; aeye 1 1 
LadamlisAi biyd eu it 


Ix | 

Pe 
Blacks 55 
Whites 59 
Hispanics 7 
Other = 1 

122 
Unknowns _78 

200 


Exam No. 


3 


4% 


44 


115 


Blacks 12 
Whites 8 
Hispanics - 
27 
Unknowns 32 
39 


Pail 


wm 


1095 (MDT-Oven Competitive) 


20% 
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Lasker, D.J, 


The Human Resources Administration (IRA), a “super- 
agency” of the City of New York, was created in 1966 in 
order to coordinate and administer the varied city pro- 
grams dealing with poverty and social services. Plantiffs 
in these two consolidated actions challenge five civil service 
examinations for positions in the Human Resources Spe- 
cialist (HRS) Series. They claim that the examinations 
had a discriminatory impact on Blacks and Hispanics and 
are not job-related. The named plaintiffs and the class 
they seek to represent are Black and Hispanic persons? 
who took and failed one or more of the five examinations 
challenged here. They seek (1) a declaration of the un- 
constitutionality of the examinations: (2) an injunction 
against appointments from the lists based on the results 
of the examinations; (3) an injunction requiring the crea- 
tion of constitutionally adequate selection procedures for 
the positions in question and (4) an injunction requiring 
the permanent appointment of those presently serving as 
provisional employees to the positions they now oceupy. 
Suit is brought under 42 U.S.C. $1981 and 1983. Jurisdie- 
tion is based on 28 U.S.C. §§1343(3) and (4), and the Fifth 
and Fourteenth Amendmerts. 

*An unusually large number of provisional employees is in- 
volved in the present suits. The large number of provisionals in the 
three titles under challenge resulted from the exhaustion of civil 
service lists based on the training and experience examination ad- 
ministered in 1968. 

In 1972 three examinations were given to select for each of the 
three titles in issue: an open competitive exam, on which both HRA 
employees and members of the publie meeting certain general qual 
ifications were eligible to compete; a promotional exam, for whic! 
only HRA employees in the next lower grade were eligible; and 
a specialty exam in Manpower Development and Training (MDT). 
As noted above, five of these exams are challenged: The open com- 
petitive exams for HRS, Sr. HRS and Sup. HRS; and the pro- 
motional exams for Sr. HRS and Sup. HRS. 


a er eee 
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The Joxes plaintiffs challenge examinations No. 163 
and 2031, the promotional and open competitive examina- 
tions for the position of Supervising Human Resources 
Specialist (Sup. HRS). The Williams plaintiffs attack the 
constitutionality of the open competitive examination (No. 
L097) for Human Resources Specialist (HRS) and beth 
the promotional and open competitive examinations for 
Senior Human Resources Specialist (Sr. HRS) (Nos. 1626 
and 1099). By earlier orders the city has been prelimi- 
narily enjoined from making appointments based on any of 
the examinations.’ 

Trial of the issues in Jones has been completed. By 
stipulation, the porties have suppleme> -d the record de- 
veloped in Jones to enable the court to decide the merits of 
Williams. 

Cases of this type, and these suits in particular, involve 
u prodigious amount of factual matter. Accordingly, we 
have so far as possible restricted the text of this opinion 
to substantive discussion, and made extensive use of foot- 


notes for other material. 
I. 


The present suits follow in the wake of several recent 
cases in this Cireuit involving eivil service examinations 


alleged to have a disparate impact on minority applicants. 
* Because permanent appoimntinent to the position of Super- 
vising HRS is a prerequisite to taking the examination for Prin- 
cipal HRS, an injunction was granted at che same time against 
holding the promotional and open competitive Principal’s exams, 
so that persons who passed the challenged Supervising HRS cxam 
and would not be deprived of the opy -ttunity of ultimately taking 
the Principal's exam on aceount ot ihe preliminary injunetion 
ugainst appointment from the Supervising HRS list. * 


“ 
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See, e.g., Vulcan Society v. Civil Service Commission, 
(hereafter “Vulean”), 490 ".2d 387 (2d Cir. 1973), aff'g¢ 360 
F. Supp. 1265 (S.D.NVY. 1973); Bridgeport Guardians, Ine. 
y, Bridgeport Civil Se e Commission, (“Bridgeport 
Guardians”), 482 F.2d 1333 (2d Cir, 1973) aff'g in part 
and rev'g in part, 354 F. Supp. 778 (D. Conn. 1973); 
Chance v. Board of Examine ¥, (“Chance”), 458 F.2d 1167 
(2d Cir. 1972) aff’g 330 F. Supp. 203 (S.D.N.Y. 1971): 
Kirkland vy. N.Y. State Dept of Correctional Services, 
(“Kirkland”), 374 F.Supp. 1361 (S.D.N.Y, 1974) 

The grou les established in those decisions require 
plaintiffs to inake a prima facie sho vine that the examina- 
tions have a “racially disproportionate impact,” Vulcan, 
490 F.2d at 391, Chance, 458 F.2d at 1175-1176: see also 
Castro v. Beecher, (“Castro”), 459 F.2d 725, 732 1st Cir 
1972). Upon such a showing the burden shifts to the defen- 
dants to establish that the challenged examinations are job- 
related, Vulcan, 490 F.2d at 391. If it is demonstrated that 
disparate examination performance results fron. the ean- 
didates’ relative qualifications for the job, rather than their 
race, the examinations are constitutionally adequate, in 
spite of their racially disparate impact. Griggs Vv. Duke 
Power Co., 401 U.S. 424 (1971 ), Chance, 330 F.Supp. at 
214. The burden on defendants is “a heavy one,” Chance. 
458 F.2d at 1176, Guardians, 482 F.2d at 1337, but is dis- 
charged if they “come forward with convincing facts estab- 
lishing a fit between the qualification aad the job.” Vulean, 
490 F.2d at 393 quoting with approval Castro, 459 F.2d at 
732. The defendants are not required to prove that no 
alternative methods of selection were available to them; 


the critical question is whether the challenged procedure is 
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constitutionally sound, not whether a better one could have 


been devised. Castro, 459 F.2d at 733, Vulcan, 49° i°.2d 


oma 


DisPROPORTIONATE IMPAC 


A. As in earlier suits, plaintiffs base their prima facie 
case on statistics provided by defendants as to the race of 
passing and failing candidates. However, as to three of 
the five examinations in question, the data is incomplete 
because HRA does not keep records of the race of can- 
didates who were not HRA employees at the time they took 
an examination. Neither side suggested or undertook, and 
the court did not order, a survey to determine the race of 


those not identified in ITRA‘’s records.’ Accordingly, as to 


‘Ye adequacy of plaintiffs’ statistical case was raised for the 


fit me in defendants’ post-trial memorandum. It was not raised 
on the motions for a preliminary injunction in Jones (September, 
1973) or Williams (December, 1973 and January, 1974); nor did 


defendants move to dismiss the complaint at the close of plaintiffs’ 
case at trial. 

We attach no particular significance to the timing of defendants’ 
attack, except to note that it comes after our decision in Hill vy. 
Human Resources Administration, 74 Civ. 1150 (March 29, 1974). 
in Which we denied a preliminary injunction because of the ap- 
parent undependability there of plaintiffs’ statistics. 

Plaintiffs suggest that our findings of statistical impact on the 
motions Tor prelin 1a Wyn TOM Wh Jones and Williams are the 
“law of the case,” by which defendants (and the court) are bound, 
Putting aside the fact thai such motions decide only the probability 
of success of the merits rather than the merits themselves, we be- 
lieve questions such as the one before us should not, after trial. be 
decided on the deliberate basis upon which preliminary relief. js 
normally predicated. 

Upon consideration of the argument raised by defendants’ post- 
trial memorandum, the court directed the parties to supplement the 
trial record by furnishing affidavits of statistical experts as to the 
significance of the statistical data in the trial record. 
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2013 the ethnicity of only 51% of the candidates is known; 
for 1097 and 1099 the figures are 54% and 60% respec- 
tively. The available statistics are set forth in the chart 
below: 


Challenged Exam No, 1631 (Sup. HRS) (Prom.) 


Pass Fail Total % Passing 
Bigeks 23.3): 12 57 69 T% 
Whites ........... 28 D4 52 54% 
Hispanies ...... 3 13 16 19% 
Unknown. ...... — | H 


43 95 138 


Challi nged Exam No. 2013 (Sup. HRS) (OC) 


Pass Fail Total % Passing 

BIRGER: 05. 39 208 247 16% 

Whites .......... 125 LOS 233 54% 

Hispanies ...... 3 17 20 15% 

Ghiners. oe 5 3 8 63% 
Subtotal .... 172 336 508 

Unknown ...... 183 303 486 38% 
355 639 994 


Challenged Exam No, 1626 (Sr. HRS) (Prom.) 


Passed Failed Total % Passing 
BIQeks: 35... 1] 51 62 18% 
Whites ...::.. 30 4 34 88% 
Hispanic ........ 3 5D 8 37% 
CGNeE vo — 2 2 
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Chall nog d Exam No. 


Blacks 
Whites 


Hispanic 


Subtotal 


Unknown 


( halli 
Blacks 
Whites 
!lispanies 
Other:.2 2 
Subtotal 


Unknown 


D0 165 

101 D+ 

R )») 

165 241 

90) 1S7 

255 428 

Vise d Ravam Na; 

)) 120 

ny) 18) 

r 1) 


Putting aside for the 


sentativeness of 


the available 


moment thy 


1099 (Sr. HRS) 


221 


1355 


683 


(HRS) 
175 
LS 


oP 
ob 


606 


(OU) 


fOG:) 


32% 


31% 


o1% 


L9% 


question of the repre- 


data for Examinations No. 


2018, 1099 and 1097, the existing figures for all five ex- 


aminations clearly indicate a disparity between the passing 


rates of white and minority candidates in excess of the 


Lad to] 


ravio which ( ‘hance ( 


held sufficient 


prima facie case, 330 F.Supp. at 210.! 


to establish a 


‘In Guardians, whites passed at 3.5 times the rate for Blacks 
and Hispanics, 354 F.Supp. at 784; in Vulcan, whites scored high 
enough to have a chance for appointment at 2.8 times the rate of 
ln Kirkland, whites 
passed at a rate of about 4 times that of Blacks and 2.5 times that 
of Hispanics. 374 F.Supp. at 1366-67, 


Blacks and Hispanics, 350 F.Supp. at 1269 
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As to No. 1631, tor which complete data is available, 
whites passed at a rate of approximately three times that 
of Blacks and Hispanics (54% to 17% and 19% respec- 
tively). On Examination No, 1626, for which the data is 
also complete, whites passed at a rate of about five times 
that of Blacks and 2.4 times the rate of Hispanies (88% 
to 18% and 37%, respectively) 

The available figures for No. 1099 indicate that whites 
passed at 2.5 the rate of Blacks and Hispanies (65% to 
26% and 25%, respectively). As to No. 2013, whites passed 
at over three times the rate of Blacks and Iispanies (54% 
to 16% and 15%, respectively). Whites passed No. 1097 
at a rate of 1.7 times that of Blacks and 2.7 that of His- 
panies (51% to 31% and 19%, respectively). In sum, the 
figures for all five examinations indicate a disparate impact 
in favor of white candidates in excees of the 1.5 to 1 ratio 
that carried the day for plaintiffs in Chance. 

Not surprisingly, defendants’ most vigorously pressed 
objection to plaintiffs’ prima facie ease is the incomplete- 
ness of the data for Nos, 2013, 1099 and 1097. They 
argue that in the absence of complete and reliable data as 
te the race and passing rate of all, or substantially all, 
candidates on these exams, plaintiffs have failed to estab- 
lish a prima facie case.” Defendants make the related argu- 

°In earlier cases, the court had before it substantially com- 
plete statistics as to the racial makeup of the group taking the tests 
In Chance the parties compiled a statistical survey to provide pass- 
fail data covering fifty examinations given over a period of seven 
years, for supervisory positions in the New York ( ‘ity school system. 
The court there did not accord significance to figures for 41 of the 
50 examinations, on which a total of only 83 minority candidates 
competed—clearly too small a sample on which to base a finding 
of disparate impact. Instead the court relied on figures from the 
nine examinations taken by ten or more minority candidates, find- 


ing that plaintiffs had established a prima facie case. 330 F.Supp. 
at 209-214. As to all 6,201 candidates taking the 50 examinations 


(footnote continued \n following page) 


330a 
Opinion Dated January 10, 1975 


ment that even assuming that the complete figures for Nos. 
1631 and 1625 show substantial disparate impact as to 


those exams, the inconclusive nature of the statisties for 


footnote continued from preceding page) 


0,910 of whom were identified by race), white candi- 
dates passed at a rate of almost 1. times the rate of minority can- 
didates (44.3% to 31 1% 

In Bridgeport Guardians vy. Bridgeport Civil Service Commis- 


in Chane ¢ 


ston, supra, Vulean Soci fy v. Civil Service Commission, supra, and 
Kirkland \ dD partimocnt of Correctional Servier Ss, Supra, statisties 
to the racial mak« up and pass-fail rates of examined groups were 
omplete or virtually complete. In Hill vy. N.Y.C Human Resources 
Administration, No. 74 Civ. 1150 Ss D.N.Y. Mareh 29, 1974 : aff'd 
I’ .2d 2d Cir, 1974), this court declined to erant a pre- 
liminary injunction in a suit challenging three examinations. As to 
the first test, data were available for only 9% of those who passed 
and 3% of those who failed. and in any event, the available sta- 
tistics did not indicate substantial disparate impact. On the second 
test the ethnic identity of 44% of passers and 16% of failures were 
known, with ‘vhites passing at a rate of about 1.5 times the rate of 
minorities. On the third test the ethnic identity of 28% of passers 
and 16% of failures were known: whites passed at about the same 
rate as Hispanics and about 1.5 times the rate of Blacks 
Defendants cite Hill as authority that anything less than com- 
plete s*atistics is too little on which to base a prima facie case on 
the m cits. We disagree with this reading on our decision there. 
First, comparison of the figures available in Hill with those now 
before the court shows that the present plaintiffs have presented 
data as to a consid rably higher proportion of candidates than those 
in Hill: as to examination No. 2013, the ethnie identity of 51% 
19% of those passing and 53% of those failing) is known; for 
Nos. 1099 and 1093 the figures are 60% (646 of those passing and 
96% of those failing and 54% (61% of those passing and 49% 
of those failing), respectively. Moreover, substantia] disparate im- 
pact was not shown, even as to the available samples, in all three 
examinations in Jill. The reverse is true in the present case be- 


cause the available samples with one exception indicate disparate 
impact of far greater magnitude than the 1.5 to ] found to be suffi- 
cient in Chance, 

Furthermore, the Hill decision merely determined a request for 
preliminary relief which classically calls for the application of dif 


(footnote continued on following page) 
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the other three tests and plaintiffs’ failure to challenge two 
additional exams in the HRS series whose results are in- 
conclusive as to impact, indicate that the class did not fare 
significantly worse chan whites on the HRS series on the 
whole, which defendants claim is the proper standard. For 
the reasons stated blow, we find that neither argument has 
merit and that plaintiffs have established a prima facie 


case as to all five exams. 


B. Although neither side produced a statistical expert 
at trial, experts for ea f the parties have submitted 
affidavits as to the sig ince of the statisties in the 
record. 


footnote continued from preceding page) 


ferent legal standards than a tris) on the merits and which, in cases 
like the present one does not perinit the deliberate consideration 
appropriate to issues of substantial public importance which jis 
possible after trial. 

In addition to Hill, defendants cite in support of their attack on 
plaintiffs prima facie case Gonzalez v. City of New York, 4 E.P.D 
"7867 (S.D.N.Y. 1972) and Boston Chaptcr, N.A.A.C.P. vy. Bee- 
cher, 7 E.P.D. §9162 (D. Mass. 1974 The cases are inapposite. 
Gonzalez involved au attack ou a civil service examination by five 
minority professional employees, four of whom failed and one of 
whom did not take the exam. The court did not have before it 
statistics as to the race of anyone who took the exam. apart from 
plaintiffs themselves. In Beeeher the race of S2% of the can- 
didates (3,181 of 3,790) was known through voluntary self-report- 
ing. 3,089 of these (97%) were whites, who passed at a rate of 
04% ; the rate for 12 candidates identifying themselves as “Ne- 
groid” was 56% ; for 18 “Blacks,” 44.5% ; for 15 Hispanics, 33%; 
and for the combined minorities 39%. In view of the small number 
of minority candidates, and a white rate exceeding the combined 
minority rate by only 1.4 to 1, the court relied not on the test re- 
sults, but on the disparity between px reentage representation of 
minorities in the genera] populatie« in the fire departments of 
various cities, in eoneluding that plaintiffs had not established a 
prima facie case 
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Plaintiffs’ expert, Richard S. Barrett, is a nationally ree- 
ognized expert in the field of testing. His affidavit sets 
forth certain computations using the Chi-Square Test, a 
generally accepted means of analyzing statisties of the type 
used in lawsuits such as this one. See Chance, 458 F.2d at 
1173, 330 F. Supp. at 212. The purpose of the Chi-Square 
Test, as described by Barrett, is to determine whether a 
differential pass rate for two or more groups arises from 
areal difference in the performance of the groups, or from 
random differences arising from chance variation in the 
sample. (Barrett affidavit, dated November 4, 1974, Para- 
sraph 4.) In this case, the Chi-Square Test attempts to 
determine whether the lower passing rates for Blacks and 
Hispanies resulted from mere chance, or from a factor 
related to race. 

Barrett’s computations, which are based on the complete 
statistics for Nos. 1631 and 1626, and on the available sta- 


tistics for Nos. 2013, 1099 and 1097, are set forth below: 


Examination ( ‘om par ISOn Ch i-Square 
1631 Black v. White ibm: 17.81 
Minority v. White _. 19.63 
2013 Black v. White 76.40 
Minority v. White... 80.42 


1626 Black v. White 2 44.60 
Minority v. White cscs “MSAD 
LO9Y Black v. White ere mee | 
Minority v. White ..... 62.49 


LOOT Black v. White . 11.49 


Minority v. White 15.32 


»9, 
oa 
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garrett states that a Chi-Square of 6.64 will occur less 
than one time in 100 as the result of chance. and that con- 
ventional statistical tables do not include values as large 
as those shown in the chart “because their occurrence as 
chance events is too small to be taken seriously.” (Barret? 
affidavit, Paragraph 6) Accordingly, as to Nos. 1631 and 
1626, for which complete statistics are available, it is 
readily apparent that plaintiffs have established dispro- 
portionate impact not resulting from chance. 

The question that is unresolved by the Chi-Square anal- 
ysis set forth above is whether the data for the entire group 
of canidates on Nos. 2013, 1097 and 1099 would show the 
same re: ults as Barrett calculated on the basis of the known 
‘andidiates on those exams. On_ this question Barrett 
states: 


“Strictly speaking such a determination can be made 
only if there is reason to believe that those whose 
identity is not known are a random sample of the total 
group. There is, of course, no way to make this deter- 
mination. However, the size of the Chi Square statis 
ties reported above [which were computed on the basis 
of the known group only] is so great that those whose 
race or ethnicity is unknown would have to differ in an 
unrealistically large degree from those whose j lentity 
is known to lead to the conclusion that the tests are 
free from adverse impact.” (Barrett affidavit. Para- 


graph 7). 


Although we recognize that in cases such as this, we may 
walk through statistical mine fields, Barrett’s conclusions 


do accord with common sense. On No. 2013, for example, 


for which the ethnicity of 51% of candidates is known (the 
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HRA population) whites passed at over 3 times the rate 
of Blacks and Hispanics. We find it distinctly improbable 
that minority group members in the non-HRA (unknown) 
group would outperform non-HRA whites on the same ex- 
amination to the extraordinary degree necessary to bring 
the overall passing rates for minorities and whites into 
rough parity. This conclusion is buttressed by Barrett’s 
observation that Nos. 2013, 1097 and 1099 are “made up of 
items of the type on which Blacks and Hispanies generally 
do more poorly than whites.” (Barrett affidavit, Paragraph 
S.) CE. Griggs, 401 U.S. at 450. We reach the same conelu- 
sion as to No. 1097, which whites passed at a rate of 1.7 
that of Blacks and 2.7 times that of Hispanics. The ethniec- 
itv and pass-fail results of 54% of the candidates are 
known. Consequently, minorities in the non-HRA group 
would have to outscore non-HRA whites substantially on 
that examination to negate the strong showing of adverse 
impact. The same conclusion applies to Examination No. 
LO99 which whites passed at a rate 2.5 that of minority 
candidates, and as to which the ethnicity of 60% of the 
candidates is known. 

In sum, we find that the data of record meets the 
standard to establish a prima facie case as articulated by 
Judge Friendly in Vulean: 


“It may well be that the cited figures and other more 


peripheral data relied on by the distriet judge did not 


As used here, the term “HRA population” indicates eandi- 
dates who were employed by HRA at the time of the examination 
and whose ethnic identity is known. The “unknown” group in Nos 
2013, 1099 and 1097 is made up or those who were not employees 
of HRA. In this Opinion they are termed the “non-HRA popu 
lation.” 


| 
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prove a racially disproportionate iimpact with complete 
mathematical certainty. But there is no requirement 
that they should. Certainty generally is illusion, and 
repose is not the destiny of man. We must not forget 
the limited office of the finding that black and Hispanic 
candidates did significantly worse in the examination 
than others. That does not at all decide the case; it 
simply places on the defendants a burden of justifiea- 
tion which they should not be unwilling to assume.” 
490 F.2d at 393.' 


The affidavit of defendants’ statistical expert, Gus W 
Grammas, is not inconsistent with our conclusions as to 
2013, 1099 and 1097.7 It states, and we agree, that neither 


The remarks of Judge Weinfeld, the trial] judge in Vulcan, 
are ilso apposite here: 


... Where publie employment practices are under challenge 
defendants usually have superior access to relevant statistical 
data than plaintiffs and . the latter will often be dependent 
on the efforts and good faith of the former, In addition, sta- 
tistical evidence by its very nature deals with probabilities 


rather than certainties All that ean be requ red of methods 
employed in gathering such evidence is that they assure rea 
sonably accurate findings. Absolute perfection usually is not 
attainable in this kind of endeavor Footnote omitted] The 
right of racial minorities to demand that the State justify even 
de facto diseriminat on may not be so restricted that it exists 
in principle but not in fact.” Vu lean, 360 F Supp. at 1270 


Grammas postulates a set ot usstumptions about the non- 
HRA populations on 2013, 1097 and 1099 Which he tests against 
the only known fact about thes populations, the overall pass-rate 
of the non-IIRA groups on each exam. Grammas’ three assum p- 
tions are (1) that the whites, Blacks ond Hispanics in the non 
HRA populations ar represented in the same proportions as the 
HRA (known) populations for exch exam; (2) that the pass-fail 
rates on each exam are the same for both the HRA and non-HRA 
groups and (3) that the non-HRA candidates are a random sutnple 


(footnote continued on following page) 
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the precise racial make-uj nor the pass-fail rates of the 
non-IIRA groups in Nos. 2013, 1097 and 1099 ean be statis- 


foutnote continued from preceding page) 
of the total population Which was eligible to take the open coin 
LOOG and 1099, 

Based on these assumptions, for which there is concededly no 
support in the record, Grammas computed hypothetical pass-fail 
rates for Blacks, whites and Hispanics in the non-HRA groups 
taking exams No. 2013, 1099 and 1097. He then compared these 


petitive exan tious, 222013, 


with the observed or real pass rates of each ethnie group on those 
eXaminations to see whether the nypothetical pass rates for the 
unknown groups accorded with the observed pass rates for the 
known group. 

As to No. 2013. based m the assumptions described above. 
Grammas con luded that the hiv p thesized and observed rates are 
the same; i.e., that it is statisti 


level of confidence) that disparate impact as to the unknown group 


Was of the same general magnitude as the impact on the known 
group 


probable (based on a 95% 


As to Nos. 1099 and 1097, for which Grammas made the same 
assumptions, the observed and hypothesized rates were signficantly 
different, leading him to r¢ ject the hypothesis that the differential 
impact was the same for the unknown group as for the known 
observed) group. However, Grammas does not indieate the de- 
gree of probability that the observed and hypothesized rates were 
the same, other than to indicate that it is something less than 


G54 probable Grammas aftidavit, | wagrapl 15) This fact of 
course leaves open the posssibility, confirmed by Barrett, that 
although it is not sta ly certain that the non-HRA groups 


showed the same disparate impact as the HRA groups, it is indeed 
likely that they would. 

There are additional reasons Why Grammas’ computations are 
of limited relevance in determining the legal question before us: 
Grammas bases his couiputations of hypothetical pass-fail rates for 
the non-HRA group on three assumptions “which plaintiffs would 
need to make in order to establish disparate impact on the non- 
HRA population.” (Grammas affidavit. Paragraph 9) This is 
clearly not the case. First, as noted above. it 


; s not necessary that 
the non-HRA population lave pr cisely the 


sane recial make-up 


as the HRA population for disparate impact to exist as to the 
non-HRA population; the rate at which minority candidates pass 
does not hinge on the number of them appeuring for the exam, 
Nor is it necessary that 


the pass-fail rates for minorities in the 


(footnote continued on following page) 
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tically inferred from the data about the HRA groups whose 
ethnicity and pass-fail rates are known because the known 
group (HRA employees) is not a random or representative 
sample of the unknown (non-IiRA) emplovees. (Grammas 
affidavit, dated November 7, 1974. Paragraphs 6-7, 20-23.) 
But that fact is not inconsistent with our conclusion. 
Strictly speaking, the precise racial make-up of the un- 
known groups in Nos. 2013, 1099 and 1097 is irrelevant; the 
issue rather is whether there is any realistic likelihood that 
non-HRA minority candidates—however many or few— 
fared well enough in comparison to non-HRA whites to 
offset the startling imbalance in favor of whites among the 
known (HRA) candidates. We conclude there is no such 
likelihood, 

Defendants’ second attack on plaintiffs’ prima facie case 
can be disposed of more easily, They claim that, notwith- 
standing plaintiffs’ prima facie showing as to the five ex- 
aminations challenged in this lawsuit, they should not be 


(footnote continued from preceding page) 


non-HRA group be shown to be substantially identical to those in 
the HRA group in order to establish disparate impact. Non-HRA 
minorities could pass the exams at far higher rates than did the 
HRA minorities and still not dissipate entirely the minimum req- 
uisite prima facie showing of 1.5 to 1 established in Chance. 

Third, it is not necessary that the non-HRA candidates were a 
“random sample of the total population which was eligible to take 
the open competitive examination.” (Grammas affidavit, Paragraph 
9) The more relevant question is whether the known HRA group 
is a reasonably representative sample of the combined HRA-non- 
HRA population that appeared for 2013. 1099 and 1097, 

In sum, in view of the startling evidence of disparate impact 
among HRA employees and the complete lack of evidence sug- 
gesting that HRA minorities perform comparatively worse relative 
to HRA whites than their non-HRA count rparts, we conclude that 
the available figures for Nos. 2013, 1099 and 1097 ar sufficient to 
establish a prima facie case. 


Opinion Dated January 10, 1975 


permitted to choose amone the exal in the HRS series, 
challenging only those in which minorities performed 
worst. Neither the facts nor the law support defendants’ 


argument, 


Of the nine examinations in the IIRS series, five are 


challenged here. Plaintiffs do not challenge the four other 
exams In the series; however, the results for three of these 
are of record: the Sr. HRS (MDT) open competitive exam 
(No. 1094), the HRS promotional exam (No. 1625), and the 
HRS (MDT) open competitive (No. 1095). The statisties 


for these are indicated in the chart below: 


Sr. HRS (MD7 ) O pre na ¢ ompel live Evam No. 1094 


Passed Fail d Total 91) Passing 


Blacks ......... LS +1 59 31% 
Whites 5 LO 15 33% 
Hispanies : 7 15 22 32% 
fo) aa : — l 1 

Subtotal .. 30 67 97 
Unknown ..... 18 78 6 19% 


48 145 193 


URS Promotional Exam No. 1625 


Passed Failed Total  % Passing 
Blacks 13 3f 49 27% 
Whites l + 5 20% 
lispanies . - 4 + 


14 44 58 
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HRS (MDT) O pe Compe lif Kram No. 1095 


Pas sé d Fail d 7 otal 
Oe Me 12 44 56 
Eee. co 8 i) 17 


Hiispanie . isons 7 16 23 
Subtotal 27 69 46 


No ethnie ! 2 74 S6 
39 143 182 


\s to 1094 and 1625, it is evident that, «Ithough the 
lesults suggest roughly equal passing rates, the samples 
are too small to be valuable. As to No. 1094. if onh one 
more white had passed, the passing rate for whites would 
rise from 33% to 40% = if two more whites had passed the 
rate would be 47%, as compared with a 31% rate for 
Blacks. These figures (476 v. 310) compare favorably 
with the 1.5 to 1 ratio in Chance. As to No. 1625, if only 
one more white had passed, the rate would be 40¢ for 
whites, as compared with 27 and 0% for Blacks and 
Hispanics, respectively. The available ficures show that as 
to 1095, only 17 whites took the exam as compared to 56 
Blacks and 23 Hispanies, but in any event whites passed at 
over Cwo tines the rate of Blacks and 1.5 the rate of His- 
pames (47% to 21% and 30%). 

Comparison of the aggregate available fieures for the 
five exams under challenge with the aggregate figures for 
all eight examinations demonstrates the sh iky factual basis 


for defendants’ arezument. 
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AGGREGATE RESULTS ON 5 EXAMS 


Passed Failed Total % Passing 
Blacks Wes: GU] 774 22% 
Whites 343 246 DSY 58% 
Hispanics . 24 110 22% 


As the chart indicates whites passed at nearly 3 times the 
rate of minorities when the challenged exams are consid- 
ered in the agereyate. 
When the results for the three examinations not chal 
; 4 


lenged by ptcintiffs are added into the aggregate compu- 


tation, the overall pass rates are not significantly altered: 


Passed Failed Total So Passing 


Blacks , 216 (32 948 23% 
Whites i 357 269 626 59% 
Hispanics 24 86 110 22% 


We regard these figures as sufficient proof that plaintiffs’ 
¢lass performed significantly worse than whites and that 
the disparity is not the result of chance, 

In any event, defendants’ argument that the plaintiffs 
should not be permitted to challenge only those exams 
Whose results show disparate impact is invalid as a matter 
of law. In Vulcan, defendants challenged plaintiffs’ sta- 
tistical case because it was based on a single examination, 
Which they claimed was insufficient to be meaningful. In 


rejecting the argument Judge Weinfeld observed: 
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“The consequence of relying upon one examination is 
only that any finding of discrimination and the relief 
to be granted will necessarily be restricted to the scope 
of the proof. The evidence presented was more than 
adequate to support a finding of discriminatory im- 
pact.” 360 F.Supp. at 1271. 


The observation applies with equal force in the case at 
hand. 


III. 


Jos RELATEDNESS 


As noted earlier, defendants have the burden of justify- 
ing the use of the challenged examinations by proving that 
they are job-related, Vulcan. 470 2d at 391, and that the 
differential impact indicated hy the statisties «esults from 
variance in qualifications for the job, rather than race. 
Griggs v. Duke Power Co., supra, 401 U.S. at 430-431. 
(1971), Chance, 330 F. Supp. at 214. This burden is dis- 
charged if the city “come[s] forward with convincing facts 
establishing a fit between the qualification and the job.” 
Vulcan, 490 Fd at 392. quoting Custro, 459 F.2d at 732: 
see also Guardians, 482 F.2d at 1337, Chance, 458 F.2d at 
1176. 


\. Case law in this Circuit recognizes three methe!. tor 
validating an exainination as job-related: criterion ' 
validation, construct validation and content val, r2 
See, e.g., Vulcan, 490 F.2d at 394-96; Guardians, 482 F.2d 
at 1557-1538 and 354 F.Supp. at 788-789; Wirkland. 374 


l.Supp. at 1370-1372. Criterion-related validation is a 


9) *) 
OtLE 


Opinion Dated January 10, 1975 


process by which relative performance on an examination 
is compared with relative performance on the job, either by 
“pre-testing” a group of current employees or by subse- 
quent on-the-job evaluation of successful candidates. See 
Vulcan, 360 F.Supp. at 1273. This method is considered 
more effective than other validation methods because it 
clearly establishes the degree of correlation between sue- 
cessful examination performance and successful job per- 
formance, Guardians, 482 F.2d at 1337 and 354 F.Supp. at 
788. However, no case in this Circuit has held that a show- 
ing of criterion validity is required for defendants to sat- 
isfy their burden of proving job-relatedness, if the test ean 
he shown to have been validated by another method. See 
Vulcan, 490 F.2d at 395. 

The second recognized method of validation is construct 
validation, which involves “identification of the general 
mentai and psychological traits believed necessary to sue- 
cessful performance of the job in question,” Vulcan, 490 
F.2d at 395, and the construction of an examination which 
tests for these qualities. Defendants do not contend that 
they validated the examinations by either criterion vali- 
dation or construct validation. 

Consequently, defendants’ proof on the issue of job-re- 
latedness hinges on whether the examinations are “content 
valid.” Judge Weinteld deseribed this method in Vulcan: 


“An examination has content validity if the content of 
the examination matches the content of the job. For 
a test to be content valid, the aptitudes and skills re- 


quired for successful examination performance must 


he those aptitudes and skills required for suecessful 
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job performance. It is essential that the examination 
test these attributes both in proportion to their relative 
importance on the job and at the level of difficulty de- 
manded by the job.’ 360 F.Supp. at 1274 (footnotes 
omitted). See also, Vulcan, 490 F.2d at 395: Guardians. 
482 F.2d at 13388: Airkland, 374 F.Supp. at 1372. 


Cases in this Circuit have recognized the difficulties of 
applying sophisticated, and unfamiliar, principles of psy- 
chometrics to jobs about which the trier of fact has only 
superficial knowledge, and have dealt with the problem on 
a pragmatic basis. Judge Friendly’s approving deseription 
of the approach Judge Weinfeld used in Vulcan sets the 
tone: 


“Instead of burying himself in a question-by-question 
analysis of Exam (0159 to determine if the test had con- 
struct or content validity, the judge noted that it was 
critical to each of the validation schemes that the exam- 
ination be carefully prepared with a keen awareness 
of the need to design questions to test for particular 
traits or abilities that had been determined to be rele- 
vant to the job. As we read his opinion, the judge 
developed a sort of sliding scale for evaluating the 
examination, wherein the poorer the quality of the test 
preparation, the greater must be the showing that 
the examination was properly job-related, and vice 
versa. This was the point he made in saying that a 
showing of poor preparation of an examination entails 
the need of ‘the most convineine testimony as to job- 


relatedness.’ The judge's approach makes excellent 


344a 


Opivion Deted January 10, 1975 


sense to us. If an examination has been badly pre- 
pared, the chance that it will turn out to be job-related 
is small. Per contra, -areful preparation gives ground 
for an inferenes, remittable to be sure, that success 
has been achieved. A principle of this sort is useful 
in lessening the burden of judicial examination-reading 
and the risk that a court will fall into error in um- 
piring a battle of experts who speak a language it 
does not fully understand. See Chance, supra, 458 F, 
2d at 1173.” 490 F.2d at 395-396. 


3. The initial step in the construction of a content-valid 
examination is the “job-analysis.” Its purpose is to ide ntify 
the knowledge, skills and abilities required for performance 
of the job. Such an analysis involves the isolation of the 
qualities most critical to job performance, an evaluation or 
weighing of their importance relative to one another, and 
a determination of the level of competence required as to 
each of them. Vulcan, 360 F.Supp. at 1274, Kirkland, 374 
F.Supp. at 1373. Obviously, the adequacy of the job analy- 
sis is crucial to a content-valid examination: unless the 
analysis accurately deseribes the “content” of the job, the 
content of the examination based on it is likely to be seri- 
ously distorted. 

Accordingly, for defendants to sustain their burden of 
proof as to the content validity of the examinations in 
issue, they must show, 


“not only that the knowledge, skills and abilities tested 


for... coincide with some of the knowledge, skills 


and abilities required successfully to perform on 
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the job, but also that 1) the attributes selected for 
examination are critical and not merely peripherally 
related to successful job performance; 2) the various 
portions of the examination are accurately weighted 
to reflect the relative importance to the job of the at- 
tributes for which they test; and 3) the level of diffi- 
culty of the exam matches the level of difficulty for 
the job.” Kirkland, 374 F. Supp. at 1372. 


Leonard Rosenberg prepared the job analysis for the 
five challenged examinations. He has been employed since 
1956 in the Department of Personnel in New York City, 
and has had varied experience in the personnel field, pri- 
marily in the area of classification of civil service titles. 
Since 1970 he has been assigned to the Bureau of Examina- 
tions, where he is responsible for all personnel matters re- 
lating to the HRS Series of Titles. Although his earlier 
work had involved a large number of “desk audits” to de- 
termine whether a particular city employee was perform- 
ing duties appropriate to his title, (Tr. 177), the job analy- 
sis for the challenged examinations* was the first he had 


undertaken for purposes of exam construction. (Tr. 230) 


* Although each examination under challenge included both 
written and oral components, and seniority of employees was given 
some weight on the promotional exams, plaintiffs concentrate their 
attack on the written portions only. Such an approach is appro- 
priate because only candidates who passed the written test were 
given the oral exam or graded on seniority. Moreover, virtually 
everyone passed the oral examination and its only effect on the 
appointment lists was to raise or lower a candidate a few places 
on the list. It appears that no member of plaintiffs’ class would 
be denied appointment as a result of the oral exam or seniority 
rating. See Vulcan, 360 F.Supp. at 1271-1272. 
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Rosenberg’s job analysis for the title of Sup. HRS*® was 
based on a series of visits to various HRA agencies and 
work locations during the period October 1-7, 1971. At the 
tune of the visits, there were about 180 Sup. HRS provi- 
sionals scattered throughout HRA. (Tr. 374) However, 
to safeguard against leakage of information relating to 
the forthcoming exams, and pursuant to city policy, Rosen- 
berg advised HRA officials that he wished to confer only 
with permanent Sup. HRS’s. (Tr. 193) At the time of 
the audit there were seven permanent Sup. HRS incum- 
bents, of whom Rosenberg interviewed four (Tr. 226). 
Beyona that he spoke to several HRA employees in higher 
titles and observed an unspecified number of HRA em- 
plovees as they went about their work. (Tr, 224) 

Defendants’ Exhibit F is the two-page written job anal- 
ysis which Rosenberg prepared on the basis of his visits.” 

By stipulation dated April 30, 1974, the parties in Williams 
agreed that, if called, the witnesses who testified on behalf of 
defendants in Jones as to the job-relatedness of examinations 1631 
and 2013 for the position of Sup. HRS would testify to the same 
effect with regard to the three examinations challenged in Williams, 
Nos. 1097, 1626 and 1099 Accordingly, we do not distinguish 
among the five examinations under challenge in our discussion of 
their preparation and job-relatedness. 

The first page lists several work locations Rosenberg visited 
and the names of the eight HRA employees (four permanent Sup. 
HRS’s and four employees in superior positions) he interviewed 
about the job of Sup. HRS. The second page lists eighteen “Ex- 
amples of Typical Tasks” which Rosenberg observed in the course 
of the audit, including, for instance, “supervision of staff,” “solve 
problems of staff,” “conduct staff meeting for control and safety 
purposes,” “negotiations for funds to permit on-the-job develop- 
ment of disadvantaged residents of poverty areas of N.Y.C.” 

Each example is followed by several Roman numerals which 
designate the particular “knowledge and skills” Rosenberg found 


(footnote continued on following page) 
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Rosenberg also prepared a one page test plan" (Defen- 
dants’ Exhibit G1) based on the job analysis. The test 
plan lists eight areas to be covered on the Sup. HRS ex- 
amination which are substantially identical to the eight 
knowledges and skills identified in the job analysis. 

For the reasons discussed below, we find that the job 
analysis and test plan prepared by the city fall short of 


(footnote continued from preceding page) 

to be required in the performance of each task The eight “knowl- 
edges and skills” include ‘ ‘Respousibilities ef H.R.A. Central and 
its constituent agencies and relevant outside agencies,” ‘“Familiar- 
ity with developments and problems in programs affecting human 
resources and the amelioration or elimi nation ot poverty,” “Ability 
to supervise, direct and develop stati.” According to the job 
analysis most of the typical tasks observed by Rosenberg require 
at least five of the listed ‘ ‘knowledges and skills”: some required 
all eight. 


In pertinent part, the test plan is as follows 


Relative 


Areas to be Covered Emphasis 

I. Function of H.R.A. its constituent agencies and 
other relevant public and private agencies 1S % 

{I. Current Developments and problems in the field 
of human services 10% 
III. Community relations and organization 10% 

IV. Judgment in on-the-job situations involving 
office services, payroll, budget and personnel 20% 

V. Techniques of staff development, supervision 
and guidance 20% 

VI. Machine, equipment and supply purchase, 
usage and management including safety 5% 

VII. Language usage including voc ‘abulary and read- 
ing comprehension 10% 

VIII. Arithmetic computations and inte rpretations of 
tables, charts, and graphs 10% 


100% 
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professional standards as delineated by the testimony and 
applicable case law. First, the evidence establishes that 
Rosenberg’s visits and interviews at work loee.‘ons of 
HRA did not eover the full specucum of tasks performed 
hy those in the title of Sup. HRS. It is undisputed that 
Rosenberg did not interview people in most of the sub- 
agencies of HRA including, for example, the Agency for 
Child Development and the Youth Services Administration 
(Tr. 285, 381; see Defendants’ Exhibit F). Consequently, 
the job analysis cannot—and on the face of it does not-—" 
purport to be a complete profile of the job title. Without 
question, the city guidelines which prevented Rosenberg 
from interviewing provisionals in the course of his visits 
to HRA made a thorough job analysis nearly impossible. 
Of close to 200 employees in the title of Sup. HRS (about 
ISO provisionals and 7 permanent incumbents), the city’s 
policy authorized Rosenberg to speak only to the seven 
permanent incumbents, and, in fact, he spoke to only four 
of these.*® 

The evidence establishes that a sample of four employees 
in the Sup. HRS title is insufficient to provide a full view 
of the job of Sup. HRS. All the witnesses agreed that it is 
difficult to imagine job titles broader than those in the 


See note 10, Supra. 


His failure to speak at least to all seven permanent Sup. 
HRS’s is unexplained. We note, however, that Rosenberg’s visits 
were arranged, at his request, by HRA officials; and that Rosenberg 
appears to have given no specific instructions regarding the number 


and variety of perso inel he wished to see (Tr. 192-195). 
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HRS Series." (see e.g. Tr. 275, 381-382. 393. 421, 425, 497- 
498, 501-502) Employees holding the generic title Sup. HRS 
may do jobs ranging from payroll and purchasing of sup- 
plies to public relations or program planning. (Defen- 
dants’ Exhibit F) in twenty different kinds of programs 
(Tr. 381). Moreover, although there are a number of small 
clusters of Sup. HRS’s who do approximately the same 
kind of work, there are no large sub-groups capable of 
easy categorization. (Tr. 382) In view of the variety of 
HR. activities, and work tasks associated with them, an in- 


sufficient interview sample would serious!y distort the over- 
all picture’ (Tr. 572-575). 

'* Some idea of the breadth of HRS’s activities will aid in under- 
standing the variety of tasks performed by those in the three gen- 
eric titles of Specialist, Supervising Specialist and Senior Special- 
ist—and the corresponding difficulty of developing job related ex- 
aminations to select for those titles. 

The purpose of HRA as established in 1966 was to effect a city- 
wide consolidation of programs in the areas of public assistance, 
social services, manpower developuient and employment and com- 
munity action. The 35 year old Department of Social Services was 
HRA’s major component. DSS included three federally-funded 
Bureaus of Public Assistance, Child Welfare and Special Services. 
In addition, the then Mayor Lindsay created, and placed within 
HRA, two other agencies: The Manpower and Career Development 
Agency, which combined all city manpower and employment ser- 
vices, and the Community Development Agency, which provided 
technical staff assistance for the Federal Community Action Pro- 
gram governed by the Council Against Poverty. 

Later additions to HRA included the Youth Services Agency, 
created in 1967, and the Agency for Child Development, a 197] 
consolidation of Headstart and City Day Care Programs. See 
Reorganization Plan for the City of New York Human Resources 
Administration (Defendants’ Exhibit H). 

*° A concrete example of such possible distortion was mentioned 
several times in the course of the trial. Although questions relat- 
ing to supervision of staff were given the maximum weight (20%) 


(footnote continued on following page) 
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Defendants called two witnesses as to the adequacy of 
the job analysis. Everett Williams is a psychologist em- 
ployed by the Educational Testing Service in Princeton, 
New Jersey. Mildred Katzell is a psychologist specializ- 
ing in the field of measurement and evaluation. Their con- 
clusion that the job analysis was professionally adequate 
(Tr. 409, 498) must be viewed in light of their eriticism 
of the small sample and the restrictive city policy which 
caused it. Katzell conceded that “it might have been de- 
sirable to have a larger sampling of the total gamut of 
the types of positions that are circumscribed by this title.” 
(Tr, 453-34) Williams testified that interviewing four out 
of seven permanent Sup. HRS’s was “very adequate in 
terms of a sample percentage,” but that “you typically 
would want to have more observation points if there are 
these wide differences [in tasks performed], usually be- 
tween 10% to 25% of the total elass.” (Tr. 498-499) Wil- 
liams made it clear that his opinion that the job analysis 
was adequate might change if the city’s restrictions on 
interviewing provisionals were lifted. (Tr. 499-500) But 


(footnote continued from preceding page) 

on the test plan defendants’ witness Harold Yourmans, Director of 
Labor Relations for IRA, testified that only 60-65% of those in 
the Sup. HRS title do any supervision at all (Tr. 341-2, 391). 
Rosenberg testified that he thought “most” Sup. HRS’s have super- 
vision responsibility, but did not supply a figure (Tr. 240-241). 
Of the four permanent incumbents he interviewed, three had duties 
involving supervision of other employees; one had little or none 
Tr. 240). Although the 20% weight Rosenberg assigned to the 
“ability to supervise, develop and direct staff” roughly reflects his 
observations, as tar as they went, it is evident that had Rosenberg 
seen. for example, one more or one less Sup. HRS with supervisory 
duties, the job analysis, test plan and examination might well have 
been weighted differently. 
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the professional and legal inadequacy of a job analysis is 
not cured simply because there is an extrinsic explana- 
tion for it, such as the city’s poliey here. In view of the 
wide variety of tasks performe those in the title of 
Sup. HRS and the large number and varied type of sub- 
agencies within HRA, it is reasonable to assume that an 
adequate sample would approach the upper end of the 
10-25% spectrum mentioned by Williams (Tr. 581-582). 


’ 


Accordingly, we regard the 2% sample used in the job 
analysis for Sup. HRS (four of a group of about 180) as 
critically insufficient. 

Defendants argue that Rosenbere’s prior experience with 
HRA inatters, his interviews with employees in titles 
higher than Sup. IZRS and his observations of many other 
employees whom he did not interview en any deficiency 
in the sample. We disagree. Assuming that Rosenberg’s 
prior experience in HRA matters gave him a general 
knowledge of the Sup. HRS title, his private knowledge 
about HRA, however extensive, cannot have been of value 
to persons constructing the examination unless committed 
to writing in the job analysis. See Kirkland, 374 F.Supp. 
at 1373-1374. But that is not the case here; on its face, 
the written job analysis purports to be based only on in- 
formation gathered from Rosenberg’s visits to HRA pur- 
suant to his assignment to prepare the particular exami- 
nations in issue. 

In any event, the value of Rosenberg’s prior experi- 
ence in personnel matters relating to HRA ought not be 
overestimated since, as noted earlier, it occurred primarily 
in the area of classification of job titles (Ir. 476-179: 212 
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) 


2). Such work demands substantially different methods 
than those required for a thorough job analysis to be used 


as the foundation of an examination (Tr. 230-231, 570-571, 


977-07). Moreover, although the job analyses Rosenberg 
prepared for the three job titles in issue were the first 


he had done ‘or an examination (Tr. 230); he spent only 
seven days to prepare the three job analyses. (Tr. 224) 
Rosenberg himself testified that a thorough job audit nor- 
mally requires from a few days to two weeks (Tr. 229), 
Finally, although it was intended that Rosenberg actually 
prepare the examinations themselves, he was reassigned 
to another position and the task fell to Helene Willingham. 
\lthough Rosenberg may have known considerably more 
about the job of Sup. HRS than the written job analysis 
discloses, Willingham never secured the benefit of his 
knowledge. On the contrary, as it turned out Rosenberg 
took no part in the construction of the exam and did not 
review it before it was administered to insure that it 
matched the job profile (Tr. 201, 205, 210, 247). 

Nor do the supplemental interviews of four employees 
in high ‘le cure the inadequacy of the sample. Plain- 
tiffs’ pert, Felix Lopez, testified that such interviews 
could not substitute for the perceptions of those holding 
the job to be tested, and that an adequate sampling of both 
categories of employees was necessary to a proper job 
analysis. (Tr. 574-575) 

The relatively casual approach which characterizes the 
sampling of the Sup. HRS population is evident in the 
written job analysis itself. A critical step in any job anal- 


ysis is the largely inferential one of breaking down an ob 
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served task into a set of component skills, abilities and 


knowledge (Tr. 521-522; 575-576) or. as Williams put it, a 
“going from some observation to a \ erbal description which 
is understandable to some set of people who will be involved 
in the act of putting together the [test].” (Tr. 525) 
Lopez testified that he would not be able to construct 
a content-valid tet on the basis of the job analysis and 
test plan. (Tr. 585) His criticisms were sensible and per- 
suasive. First, the description of ‘oth “typical tasks” and 
“knowledge and skills” yre too ambicuous and unrefined 
to give any real idea about what the job involves and what 
is required to perform it. (Tr. 570, 572, 576, 580, 585-586 : 
see also 518-519) The job a alysis does not indicate what 
level of proficiency is required as to each skill, a critical 
defeet. (Tr. 576-577, 580, 588) It does not explain how 
or why the skills in the test plan were weighted as they 
were, also a serious defect (Tr. O85-D88) ; or even suggest 
apart from the “examples of typical tasks”—that the jobs 
held by those in the title of Sup. HRS can be different from 


one another and require different abilities (Tr. 469-7] }. 


'* For example, the task “negotiation for funds to permit on-the- 
job development of disadvantaged residents of poverty areas of 
N.Y.C.” (Defendants’ Exhibit F might involve letter writing, 
attending meetings, face-to-face negotiation, legal research and 
many other skill the capsule description provides no clue as to 
which (Tr. 588). ‘1. ¢ task “Encourages Community Corporations 
to develop expertise so that they can become autonomous of C.D.A 
supervision” is similarly ambiguous. The same criticism applies 
to the “knowledge and skills required,” which include. for example, 
“ability to supervise and direc staff,” “knowledge of community 
and publie relations” and “skills in deciding problems relative to 
such on-the-job situations as office services. payroll, budget and 
personnel.” 
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Neither Rosenberg nor defendants’ experts satisfactorily 
refuted the existence of these defects in the job analysis. 
Rosenberg testified that in order to determine what knowl- 
edge or skills were essential to performance in the job, he 
“analyzed cach and every one of the functions, activities, 
Jobs, duties, and determined that they fit into certain com- 
mon areas... and as a result came up with the eight cate- 
sories that became, in effect, the test plan.” (Tr. 199-209) 
These were weighted, Rosenberg testified according to “the 
incidence of the performance of specific types of auties and 
the essential importance, the criticality of the types of de. 
cisions that would impinge [sie] on knowledge or lack of 
knowledge in the specific areas.” (Tr. 201-202) 

Although Rosenberg correctly stated the general pro- 
cedure to be follower he did not detail how it was applied 
to the job analysis for ‘up. HRS (but see Tr. 257, 407-408, 
O87-588)."" However, it is evident that no matter how well 
he applied the procedures, they could not have resulted in 
a thorough analysis. A determination that “each and every 
one of the functions, activities, job, duties” of a Sup. 
HRS fits into eight common areas is necessarily flawed 
where, as here, much of the spectrum of tasks remains 
uninvestigated. Similarly, a weighting of the relative im- 
portance of “knowledge and skills” which is hased on the 
relative frequency of their occurrence requires a sufficient 
samp! > to insure reliable measurement. Yet, as the written 
job analysis frankly acknowledges, Rosenberg listed only 
“examples of typical tasks” (emphasis supplied). 


'' In fact Rosenberg conceded that his consultation of previous 
uotices of examination played a significant part in his determina- 
tion of relative weights (Tr. 199). 
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C. Helene Willingham prepared the examinations for 
the title of Sup. HRS."* She has been with the Examining 
Division of the Department of Personne! since 1959 and has 
prepared over 200 examinations, most of them in the social 
services area (Tr. 259-60). In preparing the examination 
for Sup. HRS, she consulted the notice of examination, and 
the job analysis and test plan. However, the fact that she 
supplemented these with her own knowledge about the job 
title and ideas about what ought to go into the exam, 
(Tr. 280) and consulted the director of the Training Staff 
of HRA as .o functions and skills involved in the Sup. HRS 
titie (Tr. 279-280) suggests that she did not find Rosen- 
berg’s work sufficiently complete. For example, she felt 
that the area of the test plan relating to “ 1achine, equip- 
ment and supply purchase, usage and management” was 
weighted tor heavily because “it might have been too par- 
ticular and teo many people wouldn’t know anything about 
it.” (Tr. 291-292) 


'* The determination of what sort of examination would be ap- 
propriate for the job titles in issue appears to have been casually 
made. Rosenberg, who was to construct the tests, considered giving 
an examination based partly on “training and experience”. Such 
an exam involves rating of applicants partly according to a resume 
of their education and employment history, and partly on a normal 
written test of the multiple choice t ae (Tr. 205-206). However, 


Rosenberg appears to have left the sceue before a final choice was 
made; the record does not indicate vit made the critical decision 


as ic the kind of test to be used (Tr. 521-522 


Moreover, the decision to peg the passing score on the examina- 
tions at 70% appears to have been based on administrative con- 
venience rather than a determination tiiat euch a score separated 
those who are qualified for the job from those wio are not. See 
Kirkland, 374 F. Supp. at 1577. Rosenberg testified that the pass- 
ing grade was set at 70% a it was the normal City practice 
to do so (Tr. 248-249, see also ' 456-458, 612-616). 
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The raw material for the individual questions or “items” 
on the examination came from several sources. Willingham 
consulted “all sorts of HRA procedural material and news- 
letters” distributed to component agencies of HRA, various 
professional journals and government publications she 
thought relevant, and newspaper clippings and various 
texts. (Tr. 264-65) She also made extensive use of the 
training materials used in a course given by HRA to pre- 
pare candidates for the Sup. HRS exam. (Tr. 273-274) 

On reviewing the test plan and job audit, Willingham 
determined that some questions ought to be constructed 
by her staff and others ought to be referred to experts in 
particular fields. (Tr. 270) Accordingly, she invited two 
outside experts to submit questions dealin. with community 
organization and community relations, st: rvision and eur- 
rent events, of which nine or ten were used on the exami- 
nation. (Tr. 272-273) There is no indication that Willing- 
ham prescribed any requirements as to the level of pro- 
fic.ency or areas of concentration the questions should test. 
Indeed, it would have been difficult for her io do so, sinee 
neither the job analysis nor test plan provides any basis 
whatever for such refinements. 

Indeed a similar problem exists with regard to the ap- 
proximately seventy questions prepared by Willingham and 
h r staff.’ Although she stated that “it was certainly pos- 


'’ Of these seventy questions, Willingham based about twenty 
on materials used in an HRA training course (Tr. 274, 281) whose 
primary purpose was to prepare candidates for the examination 
(i.e., make them “test-wise’”’), rather than to make them more 
effective Supervising Human Resources Specialists (Tr. 281, 292- 
293). Although the course was free of cost and open to all pro- 
visionals (Tr, 281) there is no evidence as to how many people 
attended it (Tr. 294). 

(footnote continued on following page) 
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sible to decide on certain critical knowledges” based on 
Rosenberg’s materials (Tr. 291). Willingham did not ex- 
plain how she did so; or decided such matters as the degree 
of difficulty of the questions. However, that is beside the 
point, since the fact that the Job analysis and test plan 
needed refinement by Willingham suggests that they were 
inadequate to begin with. 

Defendants’ experts were unenthusiastie ip their ap- 
praisal of the examination. Although Williams testified 
that the test was “reasonably well put together,” both he 
and Katzell expressed reservations about the quality of 
the item construction. Katzell observed that it was “quite 
evident” that many of the myriad rules gvoverning the 
proper phrasing of questions and multiple-choice options 
on the test were violated (Tr. 411-412, 503). 

Lopez concluded the test was “poorly constructed” (Tr. 
600). In particular he noted that many questions appear 
to have more than one correct answer—even to an expert 


(footnote continued from preceding page) 

It is noteworthy, however, that the course included “orientation 
in taking multiple choice questions, how to answer graph questions, 
reading questions” (Tr. 281), and that questions on graph inter- 
pretation and reading did in fact comprise a substantial portion 
of the Sup. HRS examination that Willingham constru ‘ted: These 
questions were the very ones which Willingham considered elimi- 
nating in view of their discriminatory bias (Tr. 298) : moreover, 
there was substantial evidence that the examination favored “test- 
wise” candidates familiar with multiple choice examinations (Tr, 
297-298, 314-316, 550-553, 609-612, 621-622, see also Barrett affi- 
davit, Paragraph 8). In short, the City’s concern for total security 
with regard to the possible contents of the exam (Tr, 200, 237-238, 
281-282) appears to have boo:neranged somewhat; since HRA 
officials saw neither the test plan nor examination prior to its ad- 
ministration. They were not in any position to prevent the possible 
advantage flowing to candidates who took the training course, 
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in the field; while others suggest the proper answer to a 
test-wise candidate who may not in fact “know” the answer. 
Indeed, the record suggests, if it does not establish, that 
the exam favored those with formal education, although 
only minimum educational requirements were imposed on 
candidates. (Tr. 590-597; 600-605; 621-622, see also Tr. 


on, 


297-298, 312-516, 550-553, Barrett affidavit, Paragraph 8) 


D. The evidence as to the inadequate manner and method 
of preparation of the job analysis and the examination 
creates the “rebuttable inference” that the examination is 
not job-related. Vulean, 490 F.2d at 395-396. Although in 
cases of this type the primary emphasis is on the validity 
of the methods used in ereating the examination rather 
than the independent validity of the end product, Kirk- 
land, 374 F.Supp. at 1373, the opinion testimony as to the 
content-validity of the exam itself confirms our conclusion 
that defendants have not shown the examination to be job- 
related. 

Harold Yourman, Director of Labor Relations at HRA 
has been with the Agency since 1967. Although he ob- 
served that the exam “delves into the agency, HRA, [and] 
covers the full spectrum of HRA” (Tr. 362-363, 395) and 
is generally related to the position (Tr. 363), he expresses 


reservations about the substantial number of questions on 
supervision (Tr. 341-342, 391, 395) and conceded that the 
exam was not directly related to his earlier duties as a 
provisional Sup. HRS (Tr. 395-396). 

Katzell is coneededly not well-acquainted with the con- 
tent of the job (Tr. 427) and her conclusion that the test 
“appears to have content-validity” (Tr. 479-80, 474-477) 
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must be viewed in that light. She observed, as is obvious, 
that the questions on reading comprehension, vocabulary 
and graph interpretation related to those areas on the test 
plan (Tr. 427). However, these were the very areas that 
Willingham considered eliminating from the test, because 
of their possibly discriminatory bias (Tr. 298, see also 
Tr. 292-300) and which Lopez particularly criticized (Tr. 
608-612). As to other areas of the test, such as that deal- 
ing with knowledge of the constituent agencies of HRA. 
Katzell testified that it would he “desirable” or “appropri- 
ate” to have such knowledge (Tr. 475-477) but did not 
suggest it was critical. 

Like Katzell, Williams stopped short of stating that the 
test was content-valid, observing only that procedures used 
.* construct the test were consistent with content-validity 
(Tr. 504) and, somewhat tautologieally, that suecessful 
performance on the test certifies that a candidate possesses 
the particular knowledge being tested for (Tr. 546-549; 
see also Tr. 247, 620). Indeed, none of the witnesses was 
willing to say that the test was useful for selecting those 
who were likely to perform well on the job, which as we 
view the matter is the only reason for administering it. 
(See Tr. 247-248, 390, 449, 546, 569-70, 590, 605, 612, 620- 
621) 


Ii. As noted above, direct testimony regarding the man- 
ner of preparation and job-relatedness of the five exami- 
nations under challenge was for the most part limited to 
the two examinations (promotional and open competitive) 
for the position of Sup. HRS under attack in Jones, the 
parties stipulated that the same testimony would be given 
as to the examinations involved in Williams. Although 
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our findings as to the Sup. HRS exam require a finding 
that che other three exams (the open competitive exam for 
HRS and Senior HRS and promotional exam for Senior 
HRS) are not job related, a comparison of the job analysis, 
test plans and examinations viewed as a group fortifies this 
conclusion. 

For example, the job analyses and test plans for HRS 
and Senior HRS (Plaintiffs’ Exhibits 8 and 9) identifying 
“knowledge and skills” and “areas to be covered” that are 
almost identical to those listed in the analysis and plan for 
Sup. HRS; the relative weights assigned to the areas of 
the test are also substantially identical for all three titles. 
(Tr. 616-619) Not surprisingly, therefore, the tests based 
on these documents were very similar; Willingham, who 
prepared them, stated that the exams for HRS and Sr. 
HRS had forty questions in common (of a total of eighty), 
as did those for Sr. HRS and Sup. HRS. However, she 
sought to make the other forty questions on each test. some- 
what more difficult than those on the next lower level (Tr. 
318-320). 

The problem with Willingham’s approach is that the job 
analyses and test plans provide no basis for rational dif- 
ferentiation between the three levels to be tested. (Tr. 
618-620, 662-663) The fact that the materials prepared 
for the three titles do not distinguish to any appreciable 
extent between the nature of the jobs or the level of eom- 
petence needed to perform them confirms our conclusion 
that the examinations were not carefully prepared and, 
consequently, not job-related. 


I’, Although what we have said so far decides the case, 
it is necessary to comment further on certain factors 
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which set the present suit somewhat apart trom other 
cases of this type and which, as defendants view the mat- 
ter, support a finding of job-relatedness. 

In Chance, Vulcan, Bridgeport Guardians and Kirkland, 
the public agencies involved either had prepared no job 
analysis at all, or pieced one together from pre-existing 
documents of doubtful value for purposes of exam prepa- 
ration. Moreover, with the possible exception of Chance, 
which involved supervisory positions in the New York City 
school system, the cases deal with positions (policeman, 
fireman and correction sergeant) whose component skills 
and tasks are relatively easy to define. This combination 
of factors somewhat simplified the determination as to job- 
relatedness in earlier cases. 

The present suit does not readily fit into the mold estab- 
lished in earlier decisions. It is evident that the “Job” of 
Supervisor HRS is not a job in the same relatively restrie- 
tive sense as the job of policeman or fireman. Indeed, as 
Rosenberg acknowledged, jobs performed by individuals 
in the title of Supervising HRS may have nothing in com- 
mon with each other except salary and general level of 
responsibility (Tr. 243). Not surprisingly, therefore, de- 
fendants argue that the exams in issue pass constitutional 
muster even though they are not demonstrably related to a 
definable “job”. They contend that because HRA cannot 
in fact predict the type of work to which an individual 
might be assigned, the examinations were designed to test 
mastery of skills which Rosenberg found to be basie to all 
Jos performed by Supervising HRS’s (see Tr. 199, 202, 
240-243, 251-252, 275, 546). 

The weakness of this argument is that defendants have 
not established either that there is in fact such a core of 
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sxills common to all jobs within the extraordinarily dif- 
fuse titles in issue (see Tr. 393-395, 421-422, 470-471) or 
that Rosenberg successfully identified them. Indeed, the 
evidence suggests the contrary. To cite the most obvious 
example, the examinatioi for Sup. HRS involved twenty 
to twenty-five questions (of a total of eighty) relating to 
supervisory skill but, as noted earlier, only 60% to 65% 
of those in the title actually have supervisory responsi- 
bility (Tr. 391, 478). 

Moreover, the ten questions on the promotional exam 
for Sup. HRS relating to the internal organization of HRA 
were understandably attacked as peripheral to the duties 
of many individuals in the title; (Tr. 474) it is diffieult to 
see how such questions can be considered essential to all 
those in the title in view of the fact that the open com- 
pet.tive exam for Sup. HRS omitted these very questions 
in favor of more general questions dealing with “Functions 
of Relevant Public and Private Agencies” (T'r. 278-279). 

Indeed, if defendants are correct that the examinations 
tested skills common to all jobs within the title and were 
job-related, it is nearly past understanding why substantial 
numbers of provisionals at all three levels failed the exami- 
nation; and why the overall pass rates for the open com- 
petitive exams for Sup. HRS and HRS were higher than on 
the promotional exams for the same titles.” (See Tr. 450- 

“On the Sup. HRS exam, 31% of the candidates passed the pro- 
motional exam, while 36% passed the open competitive exam. 
On the HRS exam 24% passed the promotional exam: 33% passed 
the open competitive. 37% passed the open competitive exam for 
Sr. HRS; 41% passed the promotional exam. 

Even more significantly, only 41 out of 174 provisional Sup. 
IRS's passed the test to qualify for permanent employment in the 
position they were already performing. See plaintiffs’ memoran- 
dum in suport of application for preliminary relief, page 6, 
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457, 248, 327-328, 392-393, 615-616) Many of the provi- 
sionals who failed the exams in issue had been in their 
jobs for two years or more and, significantly, the only 
evidence in the record indicates that they were highly effee- 
tive performers.“ Despite the fact that the existence of 
large numbers of provisionals who had taken the tests pro- 
vided a unique opportunity for a concurrent validation 
study, (Tr. 529 ff.) defendants have come forward with no 
evidence to suggest that provisionals were doing an inade- 
quate job. 


IV. 


RcMEDY 


Plaintiffs seek and are entitled to declaratory and injune- 
tive relief. Accordingly, Examinations 2013, 1631. 1097, 
1099 and 1626 are declared unconstitutional and defendant; 
are enjoined from making appointments from eligible lists 
based on their results, and from terminating the provisional 
appointments of those in plaintiffs’ proposed class to their 
respective positions solely because they failed the exami- 
nations. 


*t The named plaintiffs have filed affidavits of their supervisors 
as well as HRA performance rating forms which uniformly indi- 
cate a high level of professional performance on the very jobs for 
which they were tested here. See, e.g., affidavit of Miguel Martinez, 
dated September 8, 1974; affidavit of Carolyn Gentile, dated Sep- 
tember 7, 1973, both in support of plaintiffs’ application for a 
preliminary injunction. 

Defendants’ witness, Harold Yourman, testified that the pro- 
visionals with whose work he was familiar were all competent per- 
formers (Tr. 392). 
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In addition, plaintiffs seek affirmative relief (1) requir- 
ing defendants to appoint an unspecified number of mem- 
bers of plaintiff class to the three positions “based on their 
experience, education and qualifications,” including evalua- 
tion of their performance as provisionals; (2) directing 
defendants to develop and administer either written exami- 
nations in accordance with the EEOC guidelines,* or some 
other selection process which is non-discriminatory and 
job-related; (3) establishing a temporary procedure for 
selection to the three positions while new permanent pro- 
cedures are developed or, alternatively, (4) directing the 
permanent appointment of the present provisionals to the 
jobs they now hold. 

Although the invalidation of the five examinations in 
issue authorizes the court to fashion appropriate affirma- 
tive relief, see Lowsiana v. United States, 380 U.S. 145, 154 
(1965), Guardians, 482 F.2d at 1340, the proper course is 
to defer decision as to the nature and extent of affirmative 
relief to enable defendants to respond to proposals set forth 
in plaintiffs’ post-trial brief. Accordingly, defendants are 
directed to file a memorandum on these issues within ten 
days of the filing of this Opinion, with plaintiffs to submit 
any reply within one week thereafter. 

There remains the matter of plaintiffs’ motion for a class 
action determination in both Williams and Jones. Plain- 
tiffs’ proposed class is composed of Blacks and Hispanics 
who took and failed one or more of the five challenged 
examinations; or who took and passed an examination but 


* See “Equal Employment Opportunity Commission Testing and 
Selecting Employees Guidelines,” 29 C.F.R. §1607 at §1607.5(a). 
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scored too low to be initially appointed. Defendants have 
nO Odjection to the grant of class status if the class is 
limited to those who failed an exam. However, although 
plaintiffs have satisfied the requirements of Rule 23, there 
is no need to designate a class: plaintiffs have requested 
only declaratory and injunctive relief, which will in any 
event benefit all members of the proposed class. See 
Vulcan, 360 F.Supp. at 1266-1267, note 1: Bridgeport 
Guardians, 354 F.Supp. at 783: 3B Moore. Federal Prac- 
tice 23.10-1 at 2768 (2d Ed. 1969). Ace dingly, the motion 
is denied. 

Plaintiffs’ request for an award of reasonable attorneys’ 
fees is denied. Although counsel fees were awarded in 
Kirkland, 374 F.Supp. at 1380-1382, they are not appro- 
priate in the present suit. Kirkland involved an examina- 
tion for the position of correction sergeant, whose prepara 
tion did not present the uniquely difficult problems involved 
in testing for the titles in issue here. Moreover, while in 
Kirkland there was an almost complete failure of proof 
on the issue of job-relatedness, we are impressed in the 
present case by the sincere efforts of Rosenberg and Wil. 
lingham to construct tests in accordance with the stringent 
legal standards applicable in this Circuit, however inade- 
quate the examinations proved to be. 


Submit order. 


Morris E. Lasker 


U.S.D.J. 


Dated: New York, New York 
January 10, 1975. 
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Lasker, D.J. 


Plaintiffs move, pursuant to Rule 54(b), Federal Rules 
of Civil Procedure, for an order revising our Opinion filed 
January 10, 1975, by striking from it the paragraph relating 
to plaintiffs’ request for counsel fees, (Ship Op. at 42-43), 
and substituting a statement that we reserve decision on 
that issue pending clarification of the applicable law by 
the Court of Appeals for this Cirenit and the Supreme 
Court. 

Putting aside the procedural objections to such a course, 
which are persuasively set forth in Defendants’ Memoran- 
dum in Opposition, the motion is denied. 

We do not agree with plaintiffs that the uncertain state 
of the law as to the award of counsel fees in (1983 cases 
justifies + e unusual relief requested here. Trial courts 
are regularly called upon to rme when the law is not set- 
tled; we have done so in the present case and adhere to 
the conclusion reached in our January Opinion. 

[t is true that we awarded counsel fees in Kirkland v. 
N.Y. State Department of Correctional Services, 374 F. 
Supp. 1361, 1380-1382 (S.D.N.Y. 1974). In doing so, we 
noted that the issue of availability of attorney’s fees in 
$1983 cases is “novel, at least in this Circuit,” and cata- 
logued at some length decisions of other Courts of Appeal 
which approved the award of fees in similar cases, “without 
relying on a showing of bad faith or unreasonable ob- 
duracy of defendants.” 374 F.Supp. at 1381. 

We did not express a view as to whether a trial court 
must award counsel fees in every §1983 case in which fees 


would have been awarded had the suit been brought by a 
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parallel jtuisdictional route, e.g., Title VII of the 1964 Civil 
Rights Act. Indeed, it was not necessary in Kirkland to 
determine that issue; having found . hat counsel fees may 
he awarded in $1983 cases even in the absence of bad faith, 
we awarded fees on the strength of our finding that “posi- 
tive evidence of job-relatedness is conspicuous by its ab- 
sence.” 374 F.Supp. at 1378. In short, we held only that 
a showing of bad faith was not a prerequisite to recovery 
of fees; we did not hold that defendants good faith is not 
a factor to be considered in determining whether an award 
should be granted. 

Shortly after Airkland was filed, the Court of Appeals 
furnished a pair of clues as to its view on the subject. 
Jordan vy. Fusari, 496 F.2d 646 (2d Cir. 1974), decided four 
weeks after Kirkland, was a §1983 class action in which the 
trial judge awarded attorneys fees to successful plaintiffs 
who claimed unemployment compensation benefits, the fees 
to be paid out of the recovery. On appeal, the appellee 
argued that moneys in a state’s unemployment fund are 
payable solely for unemployment benefits. Appellants dis- 
puted that contention and argued, for the first time on 
appeal, that fees were in any event recoverable on the 
private attorney general theory espoused in Newman v. 
Piggi Park Ente rprises, 390 U.S. 400, 402 (1968). Noting 
that this alternative theory had not been presented to the 
district judge, the court remanded the issue, observing that 
appellants’ new contention might ‘well justify a judgment 
imposing reasonable attorneys fees on defendant, without 


deduction from the awards to plaintiffs’ class.” 496 #.2d 
ut 650-651, 
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While Jordan indies 
cessful plaintiffs in a $1983 case of attorneys fees, it did not 
suggest under what circumstances 
awarded. The Court of Appea 
in Bridgeport Guardians, Inc. 
Comm'n, 497 F.2d 111 


ated the possible availability to sue- 


they are properly 
ls intimated those standards 
v. Bridgeport Civil Service 
3, (2d Cir. June, 1974). Appellants 
argued that the court below had abused 
refusing to award attorneys fees, 
it had discretion to award such f 


its discretion by 
The trial judge found 
ees, but denied them be- 
cause the litigation was not compel 


led by defendants’ “un- 
reasonable, obdurate obstinacy,” 


see Stolberg v. Trustees 
for the State Colleges of Connecticut, 474 F.2d 485, 490 
(2d Cir. 1973). In affirming, the 
the possibility that counsel fees might be appropriate in 
some $1983 cases,” but stated that “the failure of Congress 
to provide for such fees . ~, js significant,” 497 F.2d at 1115 
(empnasis added). Because the court found only that 
trial judge had not abused his disere 
it did not comment on the legal standard applied by the 
trial judge. However, it did note that some of the fae tors 


to be considered in determining 


vurt did not “rule out 


ue 


tion in denying fee: 


Whether an award is to 
be made are “all the facts of [the] 


case, the contrik-itio~ 
mac? by counsel for plaintiffs 


, as well as the reaso: 
ness of the resistance to the plaintiffs’ claims by the de... 
dants.” 497 F.2d at 1115. 

As we read Jordan and Bridgeport Guardians, the Court 
ot Appeals—if it holds that awards are available 


at ali in 
§1983 cases—would not award the 


m virtually as a matter 
of right, as in cases involving explicit statutory authoriza- 
tion. Following the suggestion of the Court of Appeals in 


Bridgeport Guardians, supra, we believe that the proper 
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standard for awards in $1983 cases les somewhere in be- 
tween “automatic” awards and those available only on the 
very stiff showing of defendants’ bad faith. ig 
This is the standard we applied—and intended to apply— 
in ones in declining plaintiffs’ request on the ground that 
defendanis had made reasonable efforts to comply with 
constitutional requirements which, in the field of. civil 
service testing, appear to be unusually difficult to satisfy. 


The motion is denied. 


lt is so ordered. 


Morris I. Lasker 


U.S.D.J. 


Dated: New York, New York 
March 19, 1975. 


371a 


Opinion Dated April 4, 1°75 
UNITED STATES DISTRICT COURT 
SoutHerN Disrricr or New YorK 


————— 


{CAPTION OMITTED] 
a 
Appearances: 


Covincron, Howarp, “LaGoop & Ho.uanp, Esgqs. 
15 Columbus Circle 
New York, New York 10023 


DEBORAH GREENBERG, Esq. 
JAcK GREENBERG, Esq. 
JEFFREY Minvz, Esq. 
10 Columbus Cizele 
New York, New York 10015 
Attorneys for Plaintiffs 


W. Bernarp Ricuianp, Esq. 
Corporation Counsel of the 
City of New York 

Municipal Building 
New York, New York 10007 
Attorney for Defendants 


Pavia J. Omansky, Esq. 
Assistant Corporation Counsel 


Opinion Dated April 4, 1975 


Lasker, D.J. 


3v Opinion dated January 10, 1975, we declared uncon- 
stitutional five examinations in the Human Resources Oc- 
cupational Series because they had a discriminatory impact 
on Blacks and Hispanics and were not established to be 
job-related. We deferred decision on the extent of the 
affirmative relief to be granted so that the parties might 
express their views on the issue. 

As stated in our January Opinion, plaintiffs seek and 
are entitled to a declaration of the unconstitutionality of 
the five examinations, and an injunction against the use 
of the exam results as the basis either for making appoint- 
nents to positions in the Human Resources Occupational 
Series, or terminating those presently serving in such posi- 
tions. Plaintiffs are also entitled to an order that defen- 
dants prepare either new written examinations for the 
challenged positions in accordance with EEOC guidelines, 
or develop some other selection process which is job-related 
and non-discriminatory. 

In addition, plaintiffs request an order that all provi- 
sional employees (not merely plaintiffs) in the three titles 
in issue be permanently appointed if they: (1) had served 
in their respective positions for at least six months prior 
to the commencement of these lawsuits; (2) have received 
satisfactory performance ratings from thc.c supervisors; 
and (3) took one of the challenged examinations. 

Alternatively, plaintiffs request that the defendants be 
directed to grant “acting” permanent appointments to pro- 
Visionals meeting such requirements, and not to terminate 
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them except for disciplinary reasons, pending the prepara- 
tion of new examinations, 

Defendants argue thai adoption of either of plaintiffs’ 
proposals would run afoul of the requirement of the New 
York State Civil Service Law that permanent appoint- 
ments be based on “merit and fitness,” which they claim 
the present provisionals have not established. 

Such an argument was firmly rejected in Chance, 496 
F.2d 820, 823 (2d Cir. 1974) and indeed its adoption would 
leave a court powerless to grant any sort of affirmative re- 
lief pending the preparation of new examinations. More- 
over, it is difficult to conceive of a group more “fit” for 
interim appointment than the present provisionals. (Opin- 
ion, p. 40, n. 21) 

However, that issue need not detain us because, as we 
view the matter, the nub of the disagreement between plain- 
tiffs and defendants arises from fiscal considerations rather 
than strictly legal ones. Plaintiffs argue that in a period of 
municipal job freezes and layoffs they cannot obtain truly 
effective relief unless they are at least partially shielded 
from the prospect of termination, which defendants ac- 
knowledge to be a well-founded concern. They also urge 
that if the City had not administered constitutionally in- 
adequate exams in the first instance they would already be 
permanent appointees anc, as such, far less likely to be 
terminated than provisionals. 

Although plaintiffs’ appeal to the equities has consider- 
able attraction, the proposed relief presents problems of 
its own. First, as they concede, it favors provisionals, who 
received appointment to their present positions more or 
less fortuitously, at the expense of candidates who took 
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the open competitive exams for the same positions. For 
example, in operation plaintiffs’ proposed plan would re- 
sult in the appointment of provisionals who took and failed 
the promotional exam over candidates who passed the open 
competitive exam. Moreover, because plaintiffs contem- 
plate the permanent appointment of all provisionals, and 
not merely Blacks and Hispanics, a white provisional who 
failed his exam would be appointed over a Black who 
passed the open competitive exam. In our view the divi- 
siveness which might well result is itself a strong argu- 
ment against the proposed relief. 

Moreover, the ramifications of granting permanent ap- 
pointments to provisional employees extend well beyond 
the parties and subject matter involved in the present law- 
suits. The appointment of a large number of HRA em- 
ployees by court order, at a time when the jobs of hundreds 
of other city employees may also be insecure, represents 
judicial intervention on a large scale. We have strong 
doubts as to the desirability of such a course where, as 
here, there is and can be no assurance that the particular 
provisionals involved wovld necessarily have been ap- 
pointed pursuant to a valid examination. 

The subject matter of these suits is the manner in which 
appointments must be made to the positions in issue, rather 
than the terms on which the City may terminate them. 
Because the latter issue has not been litigated and indeed 
is not even ripe for litigation, there is no basis for insulat- 
ing the present provisional employees from termination 
when other City e~~loyees have no such protection. 

Nevertheless, pi ‘ffs present a serious proposition: 
that the provisionats are likely to be terminated if layoffs 
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occur at HRA and that the sole reason why many, if not 
all of them are not in the more secure position of permanent 
employees is that the City administered invalid exami- 
nations. In such circumstances, provisionals should at least 
be given a reasonable opportunity to seek relief in the 
courts if indeed the issue ripens and HRA plans to termi- 
nate them. Accordingly, the City shall give fifteen days 
notice of its intention to terminate any plaintiff presently 
serving as a provisional employee in the three titles in 
issue. 

By our January 10 Opinion, we denied plaintiffs’ motion 
for class status. Plaintiffs renew their motion and defen- 
dants do not oppose it. Because those other than the named 
plaintiffs may have occasion to seek relief under the judg- 
ment, the motion is granted and our denial of the earlier 
motion is vacated. The class is defined as all Blacks and 
Hispanics who failed the five examinations in issue or who 
passed but scored too low to be appointed. 


Submit judgment in accordance with this memorandum. 


Morris E. Lasker 


U.S.D.J. 


Dated: New York, New York 
April 4, 1975. 


376a 


Endorsement Dated May 9, 1975 
73 Civ. 3815 


Sr 
JaMES C. Jongs, e¢ al., 


Plaintiff's, 
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THe New York Crry Human Resources 


ADMINISTRATION, et al., 


Defendants. 
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Plaintiffs, 
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THE New York Ciry Human Resources 
ADMINISTRATION, et al., 


Defendants. 


Lasker, D.J. 


Defendants move to amend our nm emorandum of April 4, 
1975 by deleting the determination of plaintiffs’ class status 
and the direction that the final judgment provide for fifteen 
days notice to any member of plaintiffs’ class presently 
serving as a provisional employee in the three titles in issue. 


Endorsement Dated May 9, 1975 


On consideration of the arguments of counsel, which they 
presented to the court with their usual enthusiasm on April 


22, 1975, the motion is disposed of as follows: 


We adhere to our conclusion that class status is appro- 
priate in these cases. See Sosna v. lowa, —— U.S. —— 
(1975), 43 U.S.L.W. 4125 (January 14, 1975), Claudia Frost 
v. W: inberger, 74-2020, Slip Op. (2d Cir. April 17, 1975) 
and cases cited there. However, the class is newly defined 
as all Blacks and Hispanics who failed any of the five ex- 
aminations in issue. 

On the City’s assurance that HRA’s procedures already 
provide and will continue to provide fourteen days notice 
to all employees whom the City plans to terminate, we 
vacate our earlier determination that the final judgement 
provide for fifteen days notice of termination. 

Defendants may submit a proposed judgment within 10 
days, together with their comments, if any, on the plaintiffs’ 
proposed judgment. 

The motion is granted to the extent indicated in this 
memorandum. 


It is so ordered. 


. 


Morris FE. Lasker 
U.S.D.J. 


\ 
Dated: New York, New York 
May 9, 1975. 
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UNITED STATES DISTR:i“T COURT 
SOUTHERN Districr or New York 


a 
{CAPTION OMITTED] 


This action having been tried to the Court without a jury, 
and the Court having made findings of fact and conclusions 
of law by opinion dated January 10, 1975, declaring Ex- 
amination Nos. 2013, 1631, 1099, 1626 and 1097, prepared 
and administered by the New York City Department of 
Personnel, for appointment within the New York City 
Human Resources Administration to the positions of Super- 
vising Human Resources Specialist, Senior Human Re- 
sources Specialist and Human Resources Specialist to be 
unconstitutional, and setting the said examinations aside: 
and the original parties hereto having filed memoranda in 
relation to the relief which should be afforded in accord- 
ance with the findings and conclusions of the Court: and 
the Court having expressed its views on the issue of relief 
by opinion dated April 4, 1975, and May 9, 1975 it is Or- 
dered, Adjudged and Decreed: 

1. Case No. 73 Civ. 3815 (Jones v. H.R.A.) and Case 
No. 74 Civ, 91 (Williams v. H.R.A.) are hereby consolidated 
for all purposes. 


2. These actions are hereby certified as class actions 


under Rule 23 of the Federal Rules of Civil Procedure, with 
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the class to bel imite to those Blacks and Hispanics who 
failed any one of the examinations heretofore adjudged to 


be invalid and unconstitutional. 


Kxamination Nos. 2013, 1631, 1099, 1626 and 1097 are 
declared invalid as violating the Constitution of the United 
States. 


4. The defendants New York City Human Resources Ad- 
ministration, New York City Department of Personnel. and 
New York City Civil Service Commission, and the named 
defendants Sugarman, Bronstein, Smith and Stadtmauer, 
and their agents, employees, and successors in office are per- 
manently enjoined from: 


(a) making permanent or provisional appointments to 
the positions of Supervising Human Resources Specialist, 
Senior Human Resources Specialist and/or Human Re- 
sources Specialist in the New York City Human Resources 
Administration based upon the results of Examination Nos. 
2013, 1631, 1097, 1626 and/or 1099 or any eligible list pro- 
mulgated pursuant to any of the said examinations: 


(b) administering or promulgating eligible lists based 
upon, or in ary way acting upon the results of Examination 
Nos. 2013, 1631, 997, 1626 and/or 1099 for the positions 
of Supervising Human Resources Specialist, Senior Human 
Resources Specialist, and/or Human Resources Specialist. 


5. The defendants, their agents, employees and succes- 
sors in office, are mandatorily enjoined to develop lawful 
non-discriminatory selection procedures for the positions 
of Supervising Human Resources Specialist, Senior Human 
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Resources Specialist and Human Resources Specialist. In 
so doing, they shall adhere to the following general guide- 
lines: 


(a) The new selection procedures shall be developed 
within the shortest practicable period; 


(b) The new selection procedures shall be developed and, 
before asage for promotional or appointment purposes, 
validated in accordance with the EEOC Guidelines on Em 
ployment Selection Procedures, 29 C.F.R. §1607.1 (1970), 


as those Guidelines are or as later revised. 


6. The deferdants shall submit to the court within thirty 
days a detailed plan fo. the development of the selection 
procedures described in Paragraph 5 hereof, and furnish a 
copy of the plan to plaintiffs. Two weeks thereafter plain- 
tiffs shall submit to the court and defense counsel in writing 
such comments as they may have with regard to the pro- 
priety of the defendants’ plan. Thereafter the court shall 
approve the plan as submitted or shall order such modifi- 
cation as it deems necessary to carry out the terms of this 
judgment. 


7. The preliminary injunction against the administra- 
tion of a promotional «> open competitive examination for 
the position of Principal Human Resources Specialist is dis- 
continued. Defendants shall publish the Notice of Examina- 
tion for the position of Principal Human Resources Special- 
ist at least 30 days before the date on which any such ex- 
amination is scheduled to be administered. 
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8. Plaintiffs’ request for the award of counsel 
plaintiffs’ attorney is denied. 


fees to 


Dated: New York, New York 


May 23, 1975. 


Morris E. Lasker 
U.S.D.J. 


Defendants’ Notice’ef Appeal in James C. Jones et al. 


UNITED STATES DISTRICT COURT 
SOUTHERN District or New York 
Index No. 73 Civ. 3815 
M.E.L. 
se 
James C. Jones, Groria De Berry, Mary J. Eccues, eé al., 
Plaintiffs, 
—against— 
THE New York Crry Human Resources 
ADMINISTRATION, et al., 


Defendants. 


SIRS: 


Novick is hereby given that the defendants hereby ap- 
peal to the United States Court of Appeals for the Second 
Circuit from the judgment entered on May 23, 1975 which 
declared the examinations herein to be invalid and this 
appeal is taken from each and every part of said judg- 
meni as well as from the whole thereof. 
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Dated: New York, N. Y. 
June 6, 1975 


Yours, etc., 


W. Bernarp Ricuianp 
Corporation Counsel 
Attorney for Defendants 

Office & P. O. Address 
Municipal Building 
New York, N. Y. 10007 


By Cart SanpDErs 
To: 


Covincron, Howarp, Hacoop & HoLLanp 
Attorneys for Plaintiffs 
15 Columbus Circle 
New York, N. Y. 1002: 


DEBORAH GREENBERG 
JACK GREENBERG 
JEFFREY Mintz 
Attorneys for Plaintiff's 
10 Columbus Circle 
New York, N. Y. 10015 


“a 
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Defendants’ Notice of Appeal in Dorothy Williams, et al. 
UNITED STATES DISTRICT COURT 
SourHeRN Disrricr or New York 
Index No. 74 Civ. 91 
M.E.L. 


pe ee 
Dorotny Witiiams, Jonn Goyco and Jounniz McCoy, et al., 


Plaintiffs, 
—against— 
THe New York Ciry Human Resources 
ADMINISTRATION, et al., 


Defendants. 


SIRs: 


Novice is hereby given that the defendants hereby up- 
peal to the United States Court of Appeals for the Second 
Cirenit from the judgment entered or Mav 23, 1975 which 
declared the examinations herein te be invalid and this 
appeal is taken from each and every part of said judgment 
as well as from the whole thereof. 
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Dated: New York, N. Y. 
June 3, 1975 


Yours, etc., 


W. Bernarp RICHLAND 
Corporation Counsel 
Attorney for Defendants 

Office & P. O. Address: 
Municipal Building 
New York, N. Y. 10007 


By Cart Sanders 


To: 


Covinecton, Howarp, Hacoop & Houanp 
Attorneys for Plaintiffs 
15 Columbus Circle 
New York, N. Y. 10023 


DrBoRAH GREENBERG 
JACK GREENBERG 
JEFFREY MINTZ 
Attorneys for Plaintiffs 
10 Columbus Circle 
New York, N. Y. 10015 
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Plaintiffs’ Notice of Appeal in James C. Jones, et al. 
UNITED STATES DISTRICT COURT 
SOUTHERN District or New York 
Index No. 73 Civ. 3815 
M.E.L. 


ee ae 
James C. Jones, Guorta De Berry, 
Mary J. Ecctss, et al., 


Plaintiffs, 


—against— 


THe New York Ciry Human Resources 
ADMINISTRATION, et all., 


Defendants. 


Sir: 


Notice is hereby given that the plaintiffs hereby appeal 
to the United States Court of Appeals for the Second Cir- 
cuit from the judgment entered herein on May 23, 1975 
which declared that the examinations herein to be invalid 
and this appeal is taken from so much of such judgment 
as demed plaintiffs’ request for the award of counsel fees 
to plaintiffs’ attorneys. 


90-7 ro 
0d/a . 


Plaintiffs’ Notice of Appeal in James C. Jones, et al. 


Dated: New York, N.Y. 
June 16, 1975 


Yours, ete., 


Covincton, Howarp, Hacoop 
& HoLianp 
15 Columb:s Cirele 
New York N.Y. 19023 
Attorney for Plaintif's 
By (IlJegitble) 
To: 
W. Bernarp RicHLanp 
Attorney for Defendants 
Office & P.O. Address: 


Municipal Building 
New York, N.Y. 10007 


DesorAH GREENBERG 
JACK GREENBERG 
JEFFREY Miytz 
Attorneys for Plaintiff's 
10 Columbus Circle 
New York, N.Y. 10015 


on 
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UNITED STATES DISTRICT COURT 
SouTHERN District or New York 
Index No. 74 Civ. 91 
M.E.L. 


—. Qa 
DorotHy WILLIAMs, Jonn Goyco and 
JounniE McCoy, et al., 


Plaintiffs, 
—against— 
THe New York Crry Human Resources 
ADMINISTRATION, e¢ al., 


Defendants. 


Sir: 


Notice is hereby given that the plaintiffs hereby appeal 
to the United States Court of Appeals for the Second Cir- 
cuit from the judgment entered herein on May 23, 1975 
which declared that the examinations herein to be invalid 


and this appeal is taken from so much of such judgment 
as denied plaintiffs’ request for the award of counsel fees 
to plaintiffs’ attorneys. 
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Dated: New York, N.Y. 
June 16, 1975 


Yours, ete., 


Covincron, Howarp, Hacoop 
& Ho tianp 
15 Columbus Circle 
New York, N.Y. 10023 
Attorneys for Plaintiffs 


By (Illegible) 


To: 


W. Bernarp RicwLanp 
Atiorney for Defendants 
Office & P.O. Address: 
Municipal Building 
New York, N.Y. 10007 


DrsorAH GREENBERG 
JACK GREENBERG 
JEFFREY MINnTz 
Attorneys for Plaintiff's 
10 Columbus Circle 
New York, N.Y. 10015 


AFFIDAVIT OF SERVICE ON ATTORNEY OF PRINTED PAPERS 


City, County and State of New York, ss.: 


“a -- 4 
being duly sworn, says, that Om the....CtS.. Kocccrccsecsnsesneececeeeseseee GAY Of.cceeececceees Bed. coens4es bcaasncaenaostviatoeercantiniaheoniveienie , 19 eS 


bee copys of the same to a person in charge of said attorney's office during the absence of said attorney therefrom, and 


leaving the same with him. 


Yo ea 
Sm” “s 
Sworn to before me, this orerater, 7S ted 1 URN I +} 
day of nme Ah fn 2 


AFFIDAVIT OF SERVICE ON ATTORNEY OF PRINTED PAPERS 
LL ES TE a SEE 


City, County and State of New York, ss.: 


“) 
being duly te Says, that on ms — =r 


“COW; AG ro! CUP) 


of the annexed oer Sa? (EMMA... Mat. Tee upon AAG 2G LO he ATT ba 
the attorney for the..f 2m 12% YES, x —~-A Gif... -uit the within entitled action by delivering 


he co of the same to a person in charge of said attorney's office during the absence of said attorney therefrom, and 


leaving the same with him. 


Sworn to before me, this Ca ; 7 ty 
day of . t/MAG —cmey 625 
cS C AIRIAM Mi “yee 


Tn ‘Nt Aavle., F cornea Ay ne 3 1969 


{ Ne 
Pi Pn bee n Expires July 1, 1977 


DUE AND TIMELY SERVICE OF 5 © tHE 
- WITHIN ° iS HEREBY ADMITTED 


THIS DAY Ci 


Attorney for 


RECORD PRESS, INC., 95 MORTON ST., NEW YORK, N. Y. 10014—{212) 243-5775 


= 38 


